Why egrep outputs two blank lines but "redirecting to file" produces actual text

Jimm Chen

4/4/24, 2:34 AM

I'm using Ubuntu 20.04.4 with stock grep version 3.4.

I find something I cannot explain, please help.

[/dev/pts/0(xterm-256color) 2023-04-04 10:28:01 ERR:0]
[chj @Ub20cppdev ~/test]
$ cat input3l.csv 
"C:\Program Files (x86)\Comodo\",223743576,223920128
"C:\Program Files (x86)\Comodo\Dragon\",223743576,223920128
"C:\Program Files (x86)\Comodo\Dragon\dragon_s.dll",127097864,127098880

[/dev/pts/0(xterm-256color) 2023-04-04 10:28:07 ERR:0]
[chj @Ub20cppdev ~/test]
$ cat input3l.csv | egrep '(".+\\"),' 
"C:\Program Files (x86)\Comodo\",223743576,223920128
"C:\Program Files (x86)\Comodo\Dragon\",223743576,223920128

[/dev/pts/0(xterm-256color) 2023-04-04 10:28:19 ERR:0]
[chj @Ub20cppdev ~/test]
$ cat input3l.csv | egrep '(".+\\"),.+' 



[/dev/pts/0(xterm-256color) 2023-04-04 10:28:33 ERR:0]
[chj @Ub20cppdev ~/test]
$ cat input3l.csv | egrep '(".+\\"),.+' > egrep1.txt

[/dev/pts/0(xterm-256color) 2023-04-04 10:28:43 ERR:0]
[chj @Ub20cppdev ~/test]
$ cat egrep1.txt 
"C:\Program Files (x86)\Comodo\",223743576,223920128
"C:\Program Files (x86)\Comodo\Dragon\",223743576,223920128

1 + 1

grep

guiverc

4/4/24, 2:38 AM

I'd apply security updates to your system asap if you're still using 20.04.4 as you say & are on-line; as a fully-upgraded 20.04 system will currently report as [20.04.6](https://fridge.ubuntu.com/2023/03/23/ubuntu-20-04-6-lts-released/) and your provided details imply you've not applied security fixes since [August 2022](https://fridge.ubuntu.com/2022/09/01/ubuntu-20-04-5-lts-released/) if not before. The links I provided are for ISO release dates, but installed systems got these updates week+ before the ISO release dates.

Score:1

Ubuntu

steeldriver

4/4/24, 10:42 AM

What you're seeing is almost certainly because of the interaction between color codes in the grep output and Windows-style (CRLF) line-endings in your input. You don't see the same when you redirect output, because the default egrep alias

$ alias egrep
alias egrep='egrep --color=auto'

only colors the output when writing to a terminal. Compare for example:

$ printf '%s\r\n' 'foobar' | grep --color=always 'foo' | sed l
\033[01;31m\033[Kfoo\033[m\033[Kbar\r$
foobar
$

versus

$ printf '%s\r\n' 'foobar' | grep --color=always 'foo.*' | sed l
\033[01;31m\033[Kfoobar\r\033[m\033[K$

$

You can see that the trailing .* (or in your case, with egrep / grep -E, trailing .+) is capturing the carriage return character \r and emitting it before the closing ANSI color sequence \033[m\033[K, causing it to overtype the matched text foo.

+ 1

Jimm Chen

4/4/24, 3:15 PM

Thank you. You just hit the nail on the head. After I save my csv with single `\n` as line separator, the weird behavior is gone.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Why egrep outputs two blank lines but "redirecting to file" produces actual text

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.