Disk read metrics not showing for postgres activity

dangel

5/16/23, 3:13 AM

I've long noticed that ESXI wasn't showing disk READ activity for my Ubuntu Postgres database server. I figured maybe it was just a weird bug with ESXI, as it would show disk WRITE activity.

I decided to install Zabbix and see if I could monitor disk read activity with that but same thing, under a very heavy DB read activity, disk read rate is 0. not close to zero, or low. but absolutely 0.

I realized that if I open a large file to transfer a file of the DB server, I DO see disk read activity then. So it's only Postgres reads not showing.

Is there some sort of way that Postgres is able to "bypass" the OS so it's not able to see\report disk read usage?

EDIT: it's also entirely possible the only write activity I'm seeing are archive files, not table writes.

0 + 0

postgresql

ubuntu

performance

hard-drive

vmware-esxi

vautee

5/16/23, 9:41 AM

We don't know how much DB traffic there is, so maybe all of that is mostly reads and possibly covered by your DB's cache? Try to do a huge bunch of inserts and updates while monitoring disk activity. Additionally: Check for I/O activity on the DB VM itself, i.e. with iostat.

Gerald Schneider

5/17/23, 12:53 PM

Most probably postgres does not need to read from disk because the requested records are cached in RAM.

dangel

5/20/23, 4:07 AM

It's possible it's pulling from ram, but one process I have runs through an entire table with a couple hundred thousand rows. it takes a while, so maybe it's pulling small enough chunks at a time to not really register. I've edit the post to attach the graph I'm paying attention to

dangel

6/1/23, 3:40 AM

so I think I figured it out. I think it's Linux caching the db files in ram. I noticed pg_dump also wasn't showing and disk activity for a roughly 3gb database. I restarted postgres, and still nothing. I then backed up another db on the same machine, and saw a ton of disk activity. this VM has 16gb of ram, and it's just a dev machine, so otherwise nothing else going on

Score:0

Server

dangel

6/1/23, 3:48 AM

This turned out to be not so much postgres caching data, but the OS itself. With 12gb of memory available and no other processes on the machine, Linux had most of the DB files cached in memory. After clearing out the ram it immediately showed high disk usage with a pg_dump

0 + 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Disk read metrics not showing for postgres activity

TH: ตัวชี้วัดการอ่านดิสก์ไม่แสดงสำหรับกิจกรรม postgres

RO: Valorile de citire a discului nu se afișează pentru activitatea postgres

RU: Показатели чтения с диска не отображаются для активности postgres

VI: Số liệu đọc đĩa không hiển thị cho hoạt động postgres

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.