Score:0

Ubuntu

Why is renaming files with sed and mv printing '$'\r'

Justin

10/28/22, 11:11 PM

I am using Ubuntu on a Windows 10 machine. I am trying to rename 5 files (file1.txt, file2.txt, file3.txt, file4.txt, file5.txt) so that each file is replaced by Sample#_VoucherID.fastq.gz.

To do this, I haver a .csv file that has a format of oldfile,newfile and I am using sed as follows"

sed 's/^/mv -vi /;s/,/ /;s/$//' < File-Rename.csv | bash -e

Whenever I run this, I instead of getting:

Sample#_VoucherID.fastq.gz for each file, I get 'Sample#_VoucherID.fastq.gz'$'\r' instead. Why is this happening? If I use mv just in the Ubuntu terminal to rename files, I get the files renamed as I want, without the ',$,and \ before and after the file name. If I don't use the | bash -e in the file and I remove the quotes, it prints:

mv -vi file1.txt Sample1_VoucherID.fastq.gz

mv -vi file2.txt Sample2_VoucherID.fastq.gz

mv -vi file3.txt Sample3_VoucherID.fastq.gz

mv -vi file4.txt Sample4_VoucherID.fastq.gz

mv -vi file5.txt Sample5_VoucherID.fastq.gz

however, it does not actually run the mv command. How can I rename my files without getting those extra characters?

162

1 + 1

command-line

bash

sed

glenn jackman

10/29/22, 12:10 AM

Because the File-Rename.csv file has DOS-style `\r\n` line endings. Use `dos2unix` on it, or do `sed -e 's/\r$//' -e 'your code here'`

Score:2

Ubuntu

steeldriver

10/29/22, 12:12 AM

This is happening because your input file File-Rename.csv has Windows-style CRLF line endings instead of Unix-style LF - the $'\r' is the shell's way of representing the carriage return character.

You can "correct" you command by changing the final sed expression from s/$// (which doesn't actually do anything - $ in a regular expression is a zero-length assertion that matches the end of the line, but doesn't actually consume a character) to s/\r$//

Alternatively, convert the input file using dos2unix

HOWEVER this approach to renaming files is problematic - in particular, it will fail if either the old or new name contains spaces or certain shell special characters - and even permits code injection¹. Instead I'd suggest something like

while IFS=, read old new; do 
  mv -vi -- "$old" "$new"
done < <(sed 's/\r$//' File-Rename.csv)

while IFS=, read old new; do 
  echo mv -vi -- "$old" "${new%$'\r'}"
done < File-Rename.csv

(remove the echo once you are happy with the proposed commands).

Note that this approach will itself fail for certain names that are legal within the CSV format - in particular those containing quoted embedded commas ("foo,bar",baz for example).

¹ _{think what happens if someone enters a filename like foo;rm * for example}

0 + 3

Justin

10/29/22, 1:15 AM

Thank you! This worked! Just for my knowledge, what did the `s/\r$\\` do? Is it saying to substitute the `CRLF` with nothing? This worked! I don't use spaces, so I used the first suggestion, but I will keep in mind the `while` code you send as well. I appreciate it!

steeldriver

10/29/22, 1:23 AM

@Justin yes that's right - `sed 's/\r$//'` matches a carriage return character `\r` anchored to the end of a line `$` and substitutes nothing in its place. You could probably safely delete *all* carriage returns (for example with `tr -d '\r'`) but the sed version is more formally correct for turning `CRLF` to `LF`

Justin

10/29/22, 5:52 PM

Thank you; that's really helpful because the times I have tried to use `dos2unix` it hasn't helped; so knowing alternative codes like you've given are great. I appreciate you taking the time to explain the code, too!

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Why is renaming files with sed and mv printing '$'\r'

TH: เหตุใดจึงเปลี่ยนชื่อไฟล์ด้วยการพิมพ์ sed และ mv '$'\r'

RO: De ce redenumirea fișierelor cu sed și mv se imprimă „$”\r”

RU: Почему переименование файлов с помощью sed и mv печатает '$'\r'

VI: Tại sao đổi tên tệp bằng sed và mv in '$'\r'

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.