Shell script runs manually, but not with cron

In my sudo crontab, edited with sudo crontab -e, I have:

# m h  dom mon dow   command
40 4 * * * /bin/bash /usr/bin/verify-miab-backup.sh

That script is:

#!/bin/bash
echo `date` > /home/mythbuntu/last_verify_miab_run.log
echo `id -u -n` >> /home/mythbuntu/last_verify_miab_run.log
export PASSPHRASE=$(cat /dozer/eandb/miab_22-04/secret_key.txt)
duplicity verify --log-file=/var/log/duplicity_backblaze_verify.log file:///dozer/eandb/miab_backblaze_backups/ /dev/null && echo "miab_last_verify_time_seconds `date +"%s"`" > /etc/grafana-text-file-collectors/miab_verify.prom

The first two lines were to help try to debug this issue. That /home/mythbuntu/last_verify_miab_run.log file contains:

Mon 10 Apr 04:40:01 BST 2023
root

so it looks like the file is being run and by root. The third line takes the secret key from a file and stores it into PASSPHRASE (is that okay in a cron job?) and then the fourth line uses that passphrase to verify a backup.

If I look at the .prom file, though:

$ cat /etc/grafana-text-file-collectors/miab_verify.prom
miab_last_verify_time_seconds 1681077406

that time corresponds to Sunday, 9 April 2023 22:56:46, which is when I last ran it manually.

Running it manually with sudo verify-miab-backup.sh, everything works as expected. /home/mythbuntu/last_verify_miab_run.log contains:

$ cat /home/mythbuntu/last_verify_miab_run.log 
Mon 10 Apr 14:10:15 BST 2023
root

and

cat /etc/grafana-text-file-collectors/miab_verify.prom
miab_last_verify_time_seconds 1681132381

is Monday, 10 April 2023 14:13:01, as you would expect.

I am guessing I am doing something obviously wrong with permissions or using variables in a way that is not permitted in a cron job, but I am just not sure where. Any help would be appreciated.

Looking at previous questions on here:

• shell script works manually, but not with cron -- I have used the Bash shebang and called with bash specifically, to make sure I'm not caught by any "bashisms".
• I've used the full path to the file as suggested here: Shell script runs fine by itself, but not in cron
• I added /bin/bash before my command Bash script runs fine, but not in cron
• I could not see anything in Why crontab scripts are not working? that would be relevant to me.

Many thanks!

I would just put the following in root's crontab:

# m h  dom mon dow   command
40 4 * * * /usr/bin/verify-miab-backup.sh

And would change the contents of your script like this:

#!/bin/bash
{
  echo -e "\n=== $(date) - $0 started - $(id -u -n) ==="
  export PASSPHRASE=$(cat /dozer/eandb/miab_22-04/secret_key.txt)
  duplicity verify ...
  echo -e "\n=== $(date) - $0 finished ==="
} &>> /home/mythbuntu/last_verify_miab_run.log

This way, all expected and not expected output (stdout) and error messages (stderr) will be appended to the end of your log file (/home/mythbuntu/last_verify_miab_run.log) and you will be able to debug your script much more easily. You should put any other commands inside the curly brackets ({}), so that you will not lose anything.

I would also determine the full path of the duplicity command (possibly /usr/bin/duplicity) and use the full path in the 5th line above, in case the process running my cron script has not included the directory containing duplicity in its PATH and I see a related error message in the log above.