Join Log in
Score:2
avatar Ubuntu

Find all files with given extension in subfolders, and add substring corresponding to subfolder

cn flag
DeltaIV

I have numerous txt files, scattered across different folders.

- case1
   |
    - 0.25
       |
        - case1.txt
    - 0.35
       |
        _ case1.txt
    - 0.30
       |
        _ case1.txt
    - 0.45
       |
        _ case1.txt

- case2
   |
    - 0.25
       |
        - case2.txt
    - 0.35
       |
        _ case2.txt
    - 0.30
       |
        _ case2.txt
    - 0.45
       |
        _ case2.txt

.
.
.

I would like to copy them all to a folder, but unfortunately, as you can see, some of them have the same name, and thus a naive find solution ends up overwriting them. I would like to copy all the txt files to a directory foo, inserting the name of the name of the subfolder they're in, before the .txt extension. Also, since that the subfolder has a dot in the name, and I need to copy these files to Windows, I'd also like to change 0.25 to 0_25. In other words, the file

- case2
   |
    - 0.25
       |
        - case2.txt

must be copied to foo as case2_0_25.txt. If a bash solution is too complex/unreadable, a Python solution would be fine too, but not a zsh one.

125
0 + 0
Score:3
bac0n avatar Ubuntu
cn flag
bac0n

You divide the copy into two transformations where the first step translates dots and slashes to underscore, and the next rearranges the matched subexpressions into the desired order.

shopt -s globstar; \
tar -cvf - --show-transformed-names --transform='s![.]!_!g' --transform='s!.*/\([^/]\+\)/\([^/]\+\)_txt$!foo/\2_\1.txt!' case*/**/*.txt | tar -xf -

Now, it's crucial --transform matches, or those files will go through unprocessed.

A Bash solution will work much the same way.

#!/bin/bash

shopt -s nullglob globstar

mkdir -p foo
while read -rd ''; do
    [[ ${REPLY//./_} =~ ([^/]+)/([^/]+)_txt$ ]] && 
        cp -va "$REPLY" "foo/${BASH_REMATCH[2]}_${BASH_REMATCH[1]}.txt"
done < <(printf %s\\0 case*/**/*.txt)
0 + 0
Score:2
terdon avatar Ubuntu
cn flag
terdon

You can do this easily enough using the globstar option of bash (from man bash):

globstar

If set, the pattern ** used in a pathname expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a /, only directories and subdirectories match.

Since we can use ** to find the files, we just need to define the new name as including the original directory names and changing . to _:

for file in **/*.txt; do 
    newName=$(sed 's|[/.]|_|g' <<<"$file" | sed 's/_txt$/.txt/')
    cp -- "$file" foo/"$newName" 
done

Explanation

  • for file in **/*.txt; do: find all files (and directories, if that's relevant) in the current directory whose name ends in *.txt.
  • newName=$(sed 's|[/.]|_|g' <<<"$file" | sed 's/_txt$/.txt/') : use sed to convert all / and . to _ in the file name. Note that $file here also includes the path, so it will be something like case1/0.25/case1.txt and that becomes case1_0_25_case1_txt. We then pass the output of the first sed to a second one which converts _txt (if found at the end of the line) to .txt, giving us ``case1_0_25_case1.txt. The final output is saved in the variable $newName`.
  • cp -- "$file" foo/"$newName": we now copy the file to the directory foo/ and with the new name. The -- is not really needed here, but ensures the approach will work with any file name, including those whose name starts with a -.

I recreated the folder structure you show in your question, ran the command above and got:


$ tree
.
├── case1
│   ├── 0.25
│   │   └── case1.txt
│   ├── 0.30
│   │   └── case1.txt
│   ├── 0.35
│   │   └── case1.txt
│   └── 0.45
│       └── case1.txt
├── case2
│   ├── 0.25
│   │   └── case2.txt
│   ├── 0.30
│   │   └── case2.txt
│   ├── 0.35
│   │   └── case2.txt
│   └── 0.45
│       └── case2.txt
└── foo
    ├── case1_0_25_case1.txt
    ├── case1_0_30_case1.txt
    ├── case1_0_35_case1.txt
    ├── case1_0_45_case1.txt
    ├── case2_0_25_case2.txt
    ├── case2_0_30_case2.txt
    ├── case2_0_35_case2.txt
    └── case2_0_45_case2.txt

11 directories, 16 files
0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.