I have a linux machine that has lots of processes in uninterruptible sleep state.
I'd like to know why this happens.
Here's part of the information of running top -o S command:
top - 14:44:51 up 298 days, 4:47, 1 user, load average: 84.54, 85.11, 85.33
Tasks: 622 total, 1 running, 621 sleeping, 0 stopped, 0 zombie
%Cpu(s): 11.5 us, 2.0 sy, 0.0 ni, 85.6 id, 0.5 wa, 0.0 hi, 0.3 si, 0.0 st
KiB Mem: 32905924 total, 28823880 used, 4082044 free, 975272 buffers
KiB Swap: 33517564 total, 8570164 used, 24947400 free. 14813544 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2153 root 20 0 22904 1156 944 D 0.0 0.0 0:00.01 ps
2425 root 20 0 4372 712 532 D 0.0 0.0 0:00.01 pidof
4148 root 20 0 4372 712 532 D 0.0 0.0 0:00.00 pidof
6608 user1 20 0 14504 844 724 D 0.0 0.0 0:00.00 ps
7103 user1 20 0 14504 828 724 D 0.0 0.0 0:00.01 ps
7672 user1 20 0 14504 844 724 D 0.0 0.0 0:00.01 ps
8081 user1 20 0 14504 772 668 D 0.0 0.0 0:00.00 ps
12816 root 20 0 22904 1168 960 D 0.0 0.0 0:00.01 ps
13720 user1 20 0 14504 884 716 D 0.0 0.0 0:00.00 ps
14854 root 20 0 22904 1172 964 D 0.0 0.0 0:00.00 ps
17040 1200 20 0 16.078g 5760 4800 D 0.0 0.0 0:00.01 msedgedriver-97
17521 user1 20 0 14504 844 724 D 0.0 0.0 0:00.00 ps
17918 user1 20 0 14504 848 724 D 0.0 0.0 0:00.01 ps
19008 root 20 0 14504 848 724 D 0.0 0.0 0:00.01 ps
19926 root 20 0 14504 852 724 D 0.0 0.0 0:00.00 ps
The D in S column means that the process is in uninterruptible sleep state.
What might cause the load average to be 84.54, 85.11, 85.33? This is too high.
Why does ps command get stuck?
Here're some information about my machine:
$ nproc
8
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.5 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.5 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
$ iostat
Linux 3.13.0-123-generic (IOT-CI) 01/12/2023 _x86_64_ (8 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
8.80 0.00 1.90 0.64 0.00 88.66
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 37.57 164.32 1690.01 4233832157 43543586647
sdb 14.96 239.74 212.36 6177083529 5471449596
sdc 3.62 22.87 35.53 589207209 915440292
dm-0 157.96 127.00 1661.97 3272293073 42821280300
dm-1 16.33 37.27 28.03 960306880 722306144
dm-2 16.33 37.27 28.03 960301900 722306144
Here's the content of /proc/2153/stack.
[<ffffffff813792b4>] call_rwsem_down_read_failed+0x14/0x30
[<ffffffff8117f332>] __access_remote_vm+0x42/0x1d0
[<ffffffff81180120>] access_process_vm+0x50/0x70
[<ffffffff8122ad2a>] proc_pid_cmdline+0x8a/0x120
[<ffffffff8122c00f>] proc_info_read+0x9f/0xf0
[<ffffffff811c2335>] vfs_read+0x95/0x160
[<ffffffff811c2e49>] SyS_read+0x49/0xa0
[<ffffffff8173dd9d>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
2153 is the process id of a ps command.
