Docker run failed: no space left on device "/dev/ptmx"

Green绿色

5/11/24, 1:47 AM

When I try to run any Docker container with an interactive terminal, I get the following error:

$ docker run -it --rm my-container
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /dev/ptmx: no space left on device: unknown.

There is still plenty of free space on my current drive and the drive on which Docker is installed. The only reference of this issue I've found so far is in this forum post, where it is suggested that the amount of pseudo terminals exceeds the threshold. I tested this on my system:

$ ls /dev/pts | wc -l
756
$ sysctl kernel.pty.max
kernel.pty.max = 4096

and I found that the amount of pseudo terminals is still far below the max value. Removing exited containers and docker system prune also didn't help. After docker system prune, the number of files in /dev/pts drops to 751, but the problem persists.

What's the issue and how to solve it?

Docker version: 20.10.17, build 100c701
System: Ubuntu 20.04

Increasing the max amount of pseudo-terminals with sudo sysctl -w kernel.pty.max=8192 does indeed mitigate this issue, but I assume, this only pushes the problem further into the future. Isn't there a more sustainable solution to this problem?

318

0 + 4

tty

docker

user1686

5/11/24, 7:08 AM

Using lsfd or lsof, can you check what processes have those ptys open, _or_ what processes have /dev/ptmx open? Raising the pty limit can be reasonable if the ptys are necessary for the software to perform the task, but it's possible that some software is _leaking_ ptys.

Green绿色

5/11/24, 8:29 AM

But I'm a bit confused about the fact that the max number of ptys is over 4000, while the current number is 700, yet this error occurs. `lsof` reveals that most ptys are ssh and tmux connections.

user1686

5/11/24, 8:58 AM

700 sounds like *a lot* of ssh and tmux sessions; are those all legitimate users or are they runaway jobs? Each container has its own /dev/pts separate from the host, so you won't see container ptys there but I believe they still add up to the system maximum.

Green绿色

5/11/24, 1:09 PM

I see. If those ptys don't show up in /dev/pts, how can I query them? Besides, when I use `lsof`, it actually shows me over 1600 applications using /dev/ptmx, 1000 of which are bash, 470 sshd and a few others. The number of users is 43.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Docker run failed: no space left on device "/dev/ptmx"

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.