How do I perform a loop on files that have the same string before the first underscore?

ke flag

I am trying to perform a loop on Ubuntu in which I concatenate two files into one single file. The directory has thousands of files, which all come in pairs that have the same string of characters before the first underscore. As an example, the directory contains the following files:

I have tried performing something along the lines of

for i in *_*.fasta \
cat $i > $i.combined.fasta

but this of course does not work, as it does not make the combined files specific to the string before the first underscore. I need to have a concatenated file of uce-1348 and another of uce-1611 (there are thousands of more files, but this is a small example of what I am looking for.

Any help would be appreciated. I have heard you can set strings and patterns, but I still don't know how to do this. Thank you!

bac0n avatar
cn flag
Is it important in which order the files are concatenated together?
ar flag

You can loop over the files as you did, but then you need to extract the start of your file name to define your output file.

Then you have to use >> to add to the output file. If you use a single > it will overwrite the content every time.

So, using your file names:

for file in *_*.fasta; do
    cat "$file" >> "$output"

The expression ${file/_*} uses Shell Parameter Expansion to delete everything after the first "_" in the file name, to produce your output file name.

cn flag

In addition to copy and move mmv can also append the contents of the source file to a target name.

mmv -a 'uce-*_*' 'uce-#1.combined.fasta'
hr flag

You need to loop over one of the patterns, and generate the other filenames using parameter expansion


for i in *; do 
  cat "$i" "${i/.nexus.phy/Sample1}" > "${i/}"

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.