I have a node alpine docker container (hosted on Azure) which periodically reaches 100% CPU usage. However, running the top
or htop
command shows no threads having high CPU usage.
How do I find out what the cause is? It seams like the node process isn't causing the high CPU usage. I also ran the node profiler and the cpu profile from the times when CPU is 100% has no obvious change to a regular profile.
htop screen shows that CPU is at 100% (mostly blue, meaning low priority threads). But the threadlist below only shows 17% usage for node.
The high CPU usage is quite periodic:
CPU usage over time.
And it started a couple days ago after an deployment, but there were no significant code changes (and reverting did not fix the issue).
Edit:
vmstat (first during normal cpu usage, 2nd during 100% CPU)
ae7d7f5ef9c8:~# vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
7 0 457816 296204 275828 1383336 4 6 167 571 0 17 33 13 54 1 0
0 0 457816 296576 275860 1383388 0 0 0 256 0 4297 30 14 56 0 0
0 0 457816 296576 275908 1383340 0 0 0 296 0 4301 23 8 69 0 0
0 0 457816 296576 275932 1383348 0 0 0 208 0 2985 22 8 70 0 0
0 0 457816 296576 275940 1383348 0 0 0 80 0 2444 21 4 75 0 0
ae7d7f5ef9c8:~# vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 0 459432 285672 259408 1331844 4 6 167 571 0 23 33 13 54 1 0
1 0 459432 247584 259420 1331860 0 0 0 124 0 4078 76 24 0 0 0
1 0 459432 122716 259468 1331988 0 0 0 300 0 5521 64 36 0 0 0
2 0 459432 282488 259504 1332020 0 0 0 212 0 5351 76 24 0 0 0
5 0 459432 228736 259596 1332024 0 0 0 340 0 3257 76 24 0 0 0