Score:1

Can't create a Docker container with a volume mounted from a user-created directory in the root of an ubuntu host (EC2)

in flag

I want to create a Docker container which has a volume mounted from a user-created directory in the root of the Docker host. The host is an Ubuntu EC2 instance.

The Error

I'd like to mount the /data directory below in a Docker container, but I get the error shown:

ubuntu:~$ ls -l /
total 120
drwxr-xr-x  25 root   root    4096 Aug 10 20:51 ./
drwxr-xr-x  25 root   root    4096 Aug 10 20:51 ../
drwxr-xr-x   2 root   root    4096 Jul 22 13:50 bin/
drwxr-xr-x   3 root   root    4096 Jul 22 13:50 boot/
drwxr-xr-x   4 ubuntu ubuntu  4096 Aug 10 19:58 data/
...
drwxr-xr-x   8 root   root    4096 May 30 12:19 home/
...
drwxr-xr-x   3 root   root    4096 Jun 20 13:39 mnt/
...
drwxr-xr-x  12 root   root    4096 Jun 12 01:03 usr/
ubuntu:~$ docker run -it --rm -v /data:/data alpine
docker: Error response from daemon: error while creating mount source path '/data': mkdir /data: read-only file system.

The same command works if I use a directory that came with the instance (e.g. /usr):

ubuntu:~$ docker run -it --rm -v /usr:/data alpine 
/ # ls /data
bin      games    include  lib      lib32    local    sbin     share    src

Additional Information

I get the same error, even if I do the following:

  1. Using --mount instead of -v, even with the readonly option.
  2. Execute the command above with sudo.
  3. Changed the permissions on / and data directory to 777
  4. Changed ownership from ubuntu to root
  5. Mount a subdirectory such as /data/subdir.

The df command indicates that there are no special mounts on the root of the instance:

ubuntu:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            7.9G     0  7.9G   0% /dev
tmpfs           1.6G  872K  1.6G   1% /run
/dev/xvda1      194G  180G   14G  93% /
tmpfs           7.9G     0  7.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           7.9G     0  7.9G   0% /sys/fs/cgroup
/dev/loop0      132M  132M     0 100% /snap/docker/796
/dev/loop2       25M   25M     0 100% /snap/amazon-ssm-agent/4046
/dev/loop3       56M   56M     0 100% /snap/core18/2128
/dev/loop1      100M  100M     0 100% /snap/core/11316
/dev/loop4      100M  100M     0 100% /snap/core/11420
/dev/loop5       56M   56M     0 100% /snap/core18/2074
/dev/loop6       34M   34M     0 100% /snap/amazon-ssm-agent/3552
s3fs            256T     0  256T   0% /mnt/s3
tmpfs           1.6G     0  1.6G   0% /run/user/1000

The docker daemon is running as root:

ubuntu:~/$ ps -ef | grep dockerd
root       964     1  1 19:49 ?        00:00:55 dockerd --group docker --exec-root=/run/snap.docker --data-root=/var/snap/docker/common/var-lib-docker --pidfile=/run/snap.docker/docker.pid --config-file=/var/snap/docker/796/config/daemon.json
root      1302     1  0 19:49 ?        00:00:01 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ubuntu    6868  2649  0 21:20 pts/1    00:00:00 grep --color=auto dockerd

Thanks in advance. This is a vexing problem and I would have thought it would be easy in Docker and AWS/EC2.

Score:1
cz flag

Note: This answer applies only to Ubuntu (and to some extent derivative distributions). It should not be applied to any other distro.

The root cause of the problem is that you have installed and run Docker as a snap, and most people should not run it this way. When a program confined by snap starts, snapd creates a container to run the program in, and only files which were accessible when the program was started will be accessible to the program. This includes your new directory /data, which does not exist in the snap container docker is running in, which is why docker tries (and fails) to create it.

The version of docker shipped with Ubuntu itself also should not be used, as it is not kept up to date. (They really ought to either keep it updated or drop it entirely, but don't count on either happening any time soon.)

Most people should use Docker from the official Docker repos to avoid a wide variety of issues that crop up with Ubuntu's docker packages.

in flag
This is very helpful. When you said "When a program confined by snap starts, snapd creates a container to run the program in", I was hoping that if I reboot, Docker will see the new directory (e.g. /data). However, that is not the case. However, it does work if I put the directory in /mnt (e.g. /mnt/data). Why is this? Is there something I can read to help me understand?
Michael Hampton avatar
cz flag
@mherzog I think snap has a specific list of directories it will expose to the contained program. But as a general rule I avoid snap.
m_highlanderish avatar
br flag
Any specific reason why you avoid snap?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.