Score:0

Server

Cannot execute nested command on remote host over ssh tunnel

Milkncookiez

5/18/24, 4:25 PM

Im running the following command in my GitLab CI job:

ssh ${REMOTE_HOST} "docker restart $(docker ps --format '{{.Names}}' | grep '^backend')"

The problem is that it returns me the following error:

/usr/bin/bash: line 156: docker: command not found
"docker restart" requires at least 1 argument.

I know that docker does exist and works on the remote host, because I can log in to the host's shell and execute docker commands myself.

Based on my understanding the error seems two be two-fold - it tells me that "docker command is not found" and then tells me that "docker restart needs at least 1 argument".

So, I assume that the docker executable is not found for the nested command, which returns the "docker: command not found" error, and since that returns no names of containers, then docker restart fails as well.

I also tried replacing docker with /usr/bin/docker inside the nested command, but to no avail.

I know the ssh tunnel works correctly because just before that command, I execute the following two commands, which work fine:

ssh ${REMOTE_HOST} "docker pull my_registry/backend:${VERSION} "
ssh ${REMOTE_HOST} "docker-compose -f docker-compose.yml up --detach --remove-orphans"

So, I really believe the issue is that the docker executable isn't found for the inner command.

1 + 0

ssh

continuous-integration

gitlab

docker

gitlab-runner

Score:3

Server

Mircea Vutcovici

5/18/24, 4:34 PM

Double quoted string will still allow local bash shell to expand the variables, command substitution,... inside the string.

This means that your command will execute docker ps --format '{{.Names}}' | grep '^backend' on the local machine, not on the remote machine as you intend.

ssh ${REMOTE_HOST} "docker restart $(docker ps --format '{{.Names}}' | grep '^backend')"

You could try escaping it with:

ssh ${REMOTE_HOST} "docker restart \$(docker ps --format '{{.Names}}' | grep '^backend')"

Or use single quotes. Here is an example:

ssh ${REMOTE_HOST} 'docker restart $(docker ps --format {{.Names}} | grep ^backend)'

More details you can find the the bash(1) man page in the QUOTING section.

   There are three quoting mechanisms: the escape character, single
   quotes, and double quotes.

   A non-quoted backslash (\) is the escape character.  It preserves
   the literal value of the next character that follows, with the
   exception of <newline>.  If a \<newline> pair appears, and the
   backslash is not itself quoted, the \<newline> is treated as a
   line continuation (that is, it is removed from the input stream
   and effectively ignored).

   Enclosing characters in single quotes preserves the literal value
   of each character within the quotes.  A single quote may not
   occur between single quotes, even when preceded by a backslash.

   Enclosing characters in double quotes preserves the literal value
   of all characters within the quotes, with the exception of $, `,
   \, and, when history expansion is enabled, !.  When the shell is
   in posix mode, the ! has no special meaning within double quotes,
   even when history expansion is enabled.  The characters $ and `
   retain their special meaning within double quotes.  The backslash
   retains its special meaning only when followed by one of the
   following characters: $, `, ", \, or <newline>.  A double quote
   may be quoted within double quotes by preceding it with a
   backslash.  If enabled, history expansion will be performed
   unless an !  appearing in double quotes is escaped using a
   backslash.  The backslash preceding the !  is not removed.

+ 3

Gerald Schneider

5/19/24, 6:10 AM

Awesome answer, just one nitpick: the braces should not be escaped. If escaped the command will not be executed but used verbatim.

Milkncookiez

5/19/24, 7:39 AM

Woah, thanks for the awesome explanation!

Mircea Vutcovici

5/19/24, 3:10 PM

Thanks @GeraldSchneider. Fixed the escaping.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Cannot execute nested command on remote host over ssh tunnel

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.