Score:0

Slow server work high iowait, raid0, cpu 100%, ram is only 20% loaded

cn flag

There is a server

  • CPU ⇢ 1x Xeon E-2236
  • RAM ⇢ 32 GB
  • HDD ⇢ 4x 10TB HDD.

The disks are assigned in a RAID0 array. This is a streaming server (nginx) designed to distribute large video files. The problem arises in the following.

When you start loading the server with traffic:

  1. grows iowait
  2. CPU utilized at 100%
  3. the server starts to get slow

What I don't understand about this is that RAM memory is not used at all. RAM is only 20% utilized.

enter image description here

At the same time, there is another server that differs only in the assembly of disks - RAID10. This server uses all RAM and the server works very well.

How can I solve this problem?


Debian 4.19 & Ubuntu 20.04

in flag
There is an insufficient amount of information here to compare two machines. Could you update the question to include the version of Ubuntu on these machines and any configuration settings? Specific details make it possible to form specific answers.
Soren A avatar
mx flag
IOWAIT occurs when the disks can't deliver data fast enough, either because you have reached the limits of the hardware or hardware errors. Verify the load (with iostat, sar, iotop or similar) and look in logs for hardware problems (in /var/log/syslog and other logs in /var/log).
Grey Hill avatar
cn flag
What configuration/specific would you like to see?
Grey Hill avatar
cn flag
Soren A, there is nothing interesting in the logs
ru flag
@GreyHill if you have no logs of hardware problems, then verify the disk loads with `iostat`, `sar`, `iotop`, or similar. As it currently stands, if you're getting IOWAIT errors you're hitting the upper limits of the system hardware's data writing/return rates. With a RAID0 if *any one* of the drives is being 'slower' than the others then that's going to be the cap of your speeds. If any one drive is having issues, then with RAID0 your entire array will be impacted even if there's no hardware errors in the logs.
Grey Hill avatar
cn flag
atop shows that www-data is in D status and no memory is used at all
Grey Hill avatar
cn flag
@Thomas Ward, I am most interested in why ram memory is not being used https://prnt.sc/15edlcy
ru flag
*Typically* applications only use RAM when they need to. Therefore, unless your application is storing a lot of data in **memory** and not on disk storage, you're not going to have a ton of RAM usage in your applications. Better that your applications are *not* consuming a ton of RAM because that'll overload your system resources. Why are you worried that your apps aren't using memory? USUALLY that's a good sign, especially if everything is working normally.
Grey Hill avatar
cn flag
@ThomasWard, please see screen https://prnt.sc/15eo39q The server on the left is working perfectly The server on the right is very slow and does not give out all the bandwidth (1 gigabit) I think due to the fact that the server does not use ram. Since nginx pseudo-streaming does not store data in memory. And how to make nginx work with memory, I cannot figure out.
Grey Hill avatar
cn flag
is reid0 so much slower than reid10?? I can not believe this.
ru flag
@GreyHill That's how NGINX is designed, it doesn't 'store everything in memory' it regularly flushes to disk. If you have a problem with how NGINX works, you'll need to bring that up with NGINX. Your graphs also don't break down *what's* using your memory, NORMALLY that's a DB or scripting process, not nginx.
Grey Hill avatar
cn flag
Do I understand you correctly that everything is working properly, and I'm worried from scratch? is reid0 so much slower than reid10?? I can not believe this. - answer please
Grey Hill avatar
cn flag
Why ssd disk work on 70-85%, and disk ssa, ssb, ssc works only on 25%?
Grey Hill avatar
cn flag
DSK | sdd | busy 86% | read 253 | write 1 | MBr/s 18.7 | MBw/s 0.0 | avio 33.3 ms |
Grey Hill avatar
cn flag
DSK | sda | busy 26% | read 512 | write 5 | MBr/s 18.5 | MBw/s 0.0 | avio 5.02 ms |
Grey Hill avatar
cn flag
DSK | sdc | busy 26% | read 528 | write 2 | MBr/s 18.5 | MBw/s 0.0 | avio 4.85 ms |
Grey Hill avatar
cn flag
DSK | sdb | busy 26% | read 526 | write 2 | MBr/s 18.4 | MBw/s 0.0 | avio 4.86 ms
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.