Score:2

Why isn't the diff command working as I expect in this Bash script?

tt flag

I'm not understanding why the diff command is not working correctly in my script. Can anybody clarify it for me?

Here is the bash script:

#!/bin/bash

SRC="src-folder"
DST="dst-folder"
FILES=( a/ b/ c/ )

for f in ${FILES[@]}; do
  cmd="diff -q  $SRC/$f $DST/$f"
  echo $cmd  # row-9
  $cmd       # row-10
done

Script output seems like this:

diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/a/ dst-folder/a/
(until buffer overflow)

When I disable/remove/comment row-10 I can see correct output (when I run these commands in an interactive shell rather than in the script I get correct results):

diff -q src-folder/a/ dst-folder/a/
diff -q src-folder/b/ dst-folder/b/
diff -q src-folder/c/ dst-folder/c/

When I change diff to rsync, everything is working correctly (call from script).

So how can I use diff in scripts? What's the reason that diff called by my script is not working as expected?

I tried also a very simple script:

#!/bin/bash

diff -q  src-folder/a dst-folder/a
diff -q  src-folder/b dst-folder/b

But it doesn't work. When I change diff command to any other (e.g. rsync), everything is working.

David avatar
cn flag
This site might and most likely will help you. https://www.geeksforgeeks.org/diff-command-linux-examples/
martin avatar
tt flag
I understand `diff` command, but I don't understand why it is not working when I call it from bash script.
David avatar
cn flag
Try a fully qualified path
martin avatar
tt flag
I tried it - the same result. The first iteration ( subfolder "a" again and again)
David avatar
cn flag
I do not see in your question where you have done that. Please post the fully qualified path example.
martin avatar
tt flag
when I change row-10 from `$cmd` to `exec $cmd` first iteration is executed (correctly) but next iteration of the loop is not started.
martin avatar
tt flag
All is executed in my $HOME folder. script is executed like this: `./show-diffs`
pLumo avatar
in flag
I guess the path is not literally `src-folder/a` but something else we don't know and you don't want tell us but which is the source of the problem. Anyways, quote your paths and the array in the for-loop and you'll be fine.
jp flag
Dan
Your script runs fine for me. The only reason I can think of is that your script is calling itself when running the `diff` command. Did you overwrite `diff` to call your own script or an alias or something like that? Change `diff` to `/usr/bin/diff` and see if you still face the issue.
pLumo avatar
in flag
@Dan, might be, but why then is it working fine in an interactive shell?
Score:1
cn flag

tl;dr: Works for me.

Long answer: When I run the script from your question on one of my test machine, it behaves exactly as expected:

a-schmidt@s-laugz2repo01:~$ ./show-diffs
diff -q src-folder/a/ dst-folder/a/
diff: src-folder/a/: No such file or directory
diff: dst-folder/a/: No such file or directory
diff -q src-folder/b/ dst-folder/b/
diff: src-folder/b/: No such file or directory
diff: dst-folder/b/: No such file or directory
diff -q src-folder/c/ dst-folder/c/
diff: src-folder/c/: No such file or directory
diff: dst-folder/c/: No such file or directory
a-schmidt@s-laugz2repo01:~$

So apparently the diff command on your system is not the same as on mine. There are several possibilities how this could happen, for example through shell aliases or local installation of a program of that name. In order to find out what it is, enter the shell command:

type diff

On an unmodified Ubuntu system this will output:

diff is /usr/bin/diff

If it outputs something else, such as "diff is aliased to ..." or "diff is a function ..." then you need to look into your shell profile to find out where that change is made, and possibly correct it.

Score:1
cn flag

I think the problem is that within your command cmd="diff -q $SRC/$f $DST/$f" there are options and after the expansion at row-10 the string contained in $cmd is not evaluated properly as a command by Bash.

When you are using the exec command the string contained in $cmd is evaluated properly. But what exec does is that it replaces the current shell with new one where the command is executed, so your loop no longer exists after the first iteration.

I've highlighted evaluated few times because in Bash have eval command which does exactly that - evaluates a string as a command. So I think the proper solution is to modify the script like this:

eval $cmd       # row-10

As a final note, if it is the entire script, why you need to assign the command to a variable which is evaluated immediately (at the next line)? I mean in this particular case you can simplify the loop in a way as this:

for f in ${FILES[@]}; do
  diff -q  "$SRC/$f" "$DST/$f"
done
in flag
Yes, I think the error lies in `cmd="diff -q $SRC/$f $DST/$f"; echo $cmd` that outputs `diff -q src-folder/a/ dst-folder/a/`, but $cmd is just a string at that point. Two things that may help, `shellcheck` and running the script with `bash -x -v myscript.sh` (and removing the quiet flag, -q, from diff). Easiest fix I think is either `cmd=$("diff -q $SRC/$f $DST/$f"); echo $cmd` where $cmd now holds the output of executing the diff (due to using -q, nothing if it matches) and deleting line 10; OR replace line 10 with "$($cmd)" to execute the string you placed in the $cmd variable.
in flag
Also, proper escaping is eg `"${FILES[@]}"` and `"$SRC"/"$f"` if you want to handle spaces in filenames and such. `shellcheck` is great for helping to identify improper escaping (you can use it [online](https://www.shellcheck.net/), or install from `sudo apt install shellcheck`.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.