Does my swapping server with ~80GB available RAM need even more?

eheu

6/2/23, 9:41 AM

I'm administering a Debian Linux x86_64 server with 125GB RAM, a 10GB swap partition and a swappiness value of 60.

A free -gw output prints:

              total        used        free      shared     buffers       cache   available
Mem:            125          20           1          25           0         103          78
Swap:             9           1           7

Every night, a cron job runs which is very disk R/W intensive and raises used memory by 2GB during its execution before returning to ~20GB.

In the same time frame, certain long-running containerized Python processes (especially gunicorn) are paged out.

With time, those processes will gradually fill the swap - after a couple of weeks, it will be 99% full.

I can clear the swap by either restarting the processes or disabling and re-enabling with swapoff and swapon. But the swap usage of the Python processes will just creep up slowly again.

I guess that part of the explanation may be, that because of the nightly cron job the Kernel is prioritizing the filesystem cache higher than the Python processes. But I also guess that some software bug in the memory handling of those Python processes may be to blame.

A friend has suggested that I purchase even more RAM so there will be more room for the filesystem cache. That seems excessive to me. I would like to more precisely diagnose the cause of the swapping and find some software solution to the problem.

So now I pass the question to Serverfault - what do you think of this phenomenon, and where should I go from here?

0 + 0

linux

memory

python

swap

Score:0

Server

AlexD

6/2/23, 9:46 AM

No, you don't need more RAM and you don't need to do swapoff/swapon. This is normal behaviour, the system detects some rarely used memory pages and moves them to the swap so it can use RAM for something more useful like file cache.

0 + 0

eheu

6/7/23, 1:47 PM

Alright, thank you. Does that fact that my swap partition is almost full strike you as an "operations smell"? Should I consider increasing the size of my swap partition? And not to cross too far into new-question territory: should there generally be a certain amount of swap unused in case of OOM killer scenarios - I guess I mean a rule of thumb?

AlexD

6/7/23, 2:35 PM

You have 78 GB of RAM available for allocation and use before you start getting OOMs, so it doesn't matter if the system uses all 10 GB of swap with 78 GB RAM available. You still can add some swap space if you get 100% swap usage so the system can swap more unused memory pages.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Does my swapping server with ~80GB available RAM need even more?

TH: เซิร์ฟเวอร์การแลกเปลี่ยนของฉันที่มี RAM ใช้ได้ประมาณ 80GB ต้องการมากกว่านี้หรือไม่

RO: Serverul meu de schimb cu ~80 GB RAM disponibilă are nevoie de și mai mult?

RU: Нужно ли моему серверу подкачки с ~80 ГБ доступной оперативной памяти еще больше?

VI: Máy chủ hoán đổi của tôi với ~80GB RAM khả dụng có cần nhiều hơn nữa không?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.