Score:0

mkdir while true loop stops at 406 folders

ar flag

Can anyone explain why my script stops making folders at 406 folders and tell me a way to get it working?

#!/bin/bash
rm -rf infinite2
while true  
do

mkdir infinite2
cd ./infinite2
mkdir infinite1
cd ./infinite1

done

I have spent a couple of days trying to figure this out and could not find why it's not working.

Soren A avatar
mx flag
It looks like you reach the maximum path-length on 4096 characters ...
waltinator avatar
it flag
What is the underlying filesystem? Check with `mount`.
Eli Thrash avatar
ar flag
How could i get more folders.Is it even possible?
Eli Thrash avatar
ar flag
The underlying file system?Would that be /
Eli Thrash avatar
ar flag
I was trying to make at least 125000000 folders
Eli Thrash avatar
ar flag
`~$ getconf -a | grep PATH_MAX PATH_MAX 4096 _POSIX_PATH_MAX 4096`
terdon avatar
cn flag
Please [edit] your question and give us some context. Do you really need to make a path that is 125000000 deep, or do you just want 125000000 folders? I mean, does it really need to be `d1/d2/d3/.../f125000000` in a single directory tree or would you be OK with `parentFolder/dir1`, `parentFolder/dir2` ... `parentFolder/dir125000000`? Even better, why do you need this? If you explain what you want to actually test/achieve, we might be able to give you a better approach for doing it.
FedKad avatar
cn flag
As @terdon already commented, this is a typical https://en.wikipedia.org/wiki/XY_problem , so I vote to close this question!
Eli Thrash avatar
ar flag
I just want to make ```parentFolder/dir1, parentFolder/dir2 ... parentFolder/dir125000000```
terdon avatar
cn flag
@EliThrash then please edit and explain that because, even if it worked, that isn't what your script would have created. Sounds like you just want `mkdir parentFolder/dir{1..125000000}`, which will probably fail because you will pass the MAX_ARG value for maximum arguments in the command line, or crash your machine as it runs out of RAM trying to calculate that brace expansion, but can work as a starting point. We just need to know what you really want.
Score:3
cn flag

In your original question, you try to create a file hierarchy like this:

.
├── infinite2
│   ├── infinite1
...
│ (endless level deep)  ├── infinite2

and your script starts getting errors when your "depth" reaches the 406th "level". There is no known file system that can hold so many levels of sub-directories as the ones you want.

However, in your comments, it seems that you want a directory hierarchy like this:

.
├── dir000000001
├── dir000000002
...
├── dir124999999
└── dir125000000

Although it may be possible to create such a huge number of single level sub-directories under a directory using a script like this:

#!/bin/bash
let i=0
while (( i < 125000 )) ; do
  mkdir $(printf "dir%06d" $i){000..999}
  let i++
done

it would be very very very (did I say enough times very?) inefficient to create and use them. I was able to test this script by creating more than two million directories; but 125 million directories are way too many.

One better alternative would be to create a three-level structure, each level holding a thousand sub-directories in a hierarchy like this:

.
├── dir000
│   ├── 000
│   │   ├── 000
│   │   ├── 001
│   │   ├── 002
...
│   │   └── 999
│   ├── 001
│   │   ├── 000
│   │   ├── 001
...
│   │   └── 999
...
│   ├── 999
│   │   ├── 000
...
│   │   └── 999
...
├── dir124
│   ├── 000
...
│   ├── 999
│   │   ├── 000
...
        └── 999

The following script can be used to create these:

#!/bin/bash
let i=0
while (( i < 125000 )) ; do
  let a=i/1000
  let b=i%1000
  mkdir -p $(printf "dir%03d/%03d/\n" $a $b){000..999}
  let i++
done

Even in this case, it would be very very very difficult to use so many directories.

Another problem is that you may easily run out of inodes in your file system: You can get the infamous No space left on device error, while your current file system does have storage space, but no inode space is left to create a new file or directory. Please, check the IFree column in the df -i . command output, before running the above script.

So, again, I think this is a typical XY problem that would require a completely different approach.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.