Score:1

Piping sudo through another command adds excess whitespace

US flag
user1676974

Background

I wanted to identify if <complex command pipeline> outputs any non-printable characters (it must be run with sudo), so I piped the output through od and was surprised to see each line of output from od had additional spaces.

System

Ubuntu 18.04 (stuck at this version sadly), gnome-shell 3.28.4, gnome terminal 3.28.2 (also putty 0.76) and bash 4.4.20

Issue

I can reproduce the issue using a single pipe from cat to cat as follows:

me@ubuntu:~$ sudo cat test | cat
this
    is a
        test
            here is more data

The same weird indenting happens with od, however if I pipe the od output (which has spurious whitespace) through cat I end up with normal output left aligned (excepting the whitespace od actually prints normally), but if I pipe the output above through another cat the spurious whitespace remains.

me@ubuntu:~$ sudo cat test | od -atx1  # broken
0000000   t   h   i   s  nl   i   s  sp   a  nl   t   e   s   t  nl   h
                                                                                74  68  69  73  0a  69  73  20  61  0a  74  65  73  74  0a  68
                                                              0000020   e   r   e  sp   i   s  sp   m   o   r   e  sp   d   a   t   a
                                                              65  72  65  20  69  73  20  6d  6f  72  65  20  64  61  74  61
                                            0000040  nl
         0a
0000041

me@ubuntu:~$ sudo cat test | od -atx1 | cat  # fixed
0000000   t   h   i   s  nl   i   s  sp   a  nl   t   e   s   t  nl   h
         74  68  69  73  0a  69  73  20  61  0a  74  65  73  74  0a  68
0000020   e   r   e  sp   i   s  sp   m   o   r   e  sp   d   a   t   a
         65  72  65  20  69  73  20  6d  6f  72  65  20  64  61  74  61
0000040  nl
         0a
0000041

me@ubuntu:~$ sudo cat test | cat | cat  # not fixed
this
    is a
        test
            here is more data

Expectations

I can't find on searching for "sudo pipe" or "sudo whitespace" any explanation for this seeming interaction between running sudo and piping the output. The following commands all output exactly as I would expect:

me@ubuntu:~$ cat test | od -atx1
0000000   t   h   i   s  nl   i   s  sp   a  nl   t   e   s   t  nl   h
         74  68  69  73  0a  69  73  20  61  0a  74  65  73  74  0a  68
0000020   e   r   e  sp   i   s  sp   m   o   r   e  sp   d   a   t   a
         65  72  65  20  69  73  20  6d  6f  72  65  20  64  61  74  61
0000040  nl
         0a
0000041

me@ubuntu:~$ cat test | cat
this
is a
test
here is more data

me@ubuntu:~$ sudo cat test
this
is a
test
here is more data

Question

How can I prevent sudo \<command1> | \<command2> from adding whitespace to the beginning of each line of output that doesn't seem to come from either of sudo or <command1> or <command2>

Additional debug

me@ubuntu:~$ stty -a
speed 38400 baud; rows 24; columns 80; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 -hupcl -cstopb cread -clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff
-iuclc -ixany -imaxbel iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke -flusho -extproc

me@ubuntu:~$ sudo stty -a
speed 38400 baud; rows 24; columns 80; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>;
eol2 = <undef>; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; discard = ^O; min = 1; time = 0;
-parenb -parodd -cmspar cs8 -hupcl -cstopb cread -clocal -crtscts
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff
-iuclc -ixany -imaxbel iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt
echoctl echoke -flusho -extproc

If I pipe the sudo stty -a command through cat, it does not change the output, other than to add the extra whitespace to the start of each line. I don't know if this helps, but apparently redirecting the output of cat (&>filename) to a file, the file has no spurious whitespace.

If I escalate to root using sudo bash (or sudo -i) the output is as I expect:

me@ubuntu:~$ sudo bash

root@ubuntu:/home/me# cat test | cat
this
is a
test
here is more data

It seems that cat is the same between my user and root:

me@ubuntu:$ type cat
cat is hashed (/bin/cat)

me@ubuntu:$ sudo bash

root@ubuntu:/home/me# type cat
cat is /bin/cat
hr flag
It *looks* like `sudo` is changing the terminal's line settings (in particular, turning off `onlcr`) - do the outputs of `stty -a` and `sudo stty -a` look different?
Raffa avatar
jp flag
I can't see how `sudo` might be the culprit here or the pipe for this matter ... But, I can imagine that `cat` might be aliased or different somehow under `root`'s environment ... Probably comparing the output from inside a `root`'s shell e.g. like `sudo bash` and then just `cat test | cat` without sudo might help.
hr flag
OK thanks for the update - it certainly has the symptoms of incorrect newline translation by the terminal (you can mimic the issue with `stty -onlcr; cat test; stty onlcr` or `stty -opost; cat test; stty opost` for example). See this related issue with ssh [Why is this binary file transferred over "ssh -t" being changed?](https://unix.stackexchange.com/questions/151916/why-is-this-binary-file-transferred-over-ssh-t-being-changed/151963#151963). BTW which terminal emulator are you using?
Score:0
je flag

You can try preceding the cat line with

stty -nl

That last character is a lower case "L" and not a numeral "1". This sets the terminal to recognize a newline character (implicit or explicit) as carriage-return plus line-feed.

I've had a similar problem to yours on upgrading to 22.04. This workaround works for me in some cases, but not all, and doesn't answer the question of how the communication with the terminal became set away from the standard in the first place. I tried it in a script with your code, and it does work in that case. It shouldn't be necessary, but, as I said, it's a workaround.

FWIW, if you want to set the terminal back again to the line-feed-only state, use the command

stty -onlcr
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.