My server (running Centos Stream release 8) stopped responding to http yesterday evening. I tried to SSH in to it this morning, with no luck, just an empty hung PuTTy screen. I did a automatic reset through my webhost's control panel (Hetzner), and after a few minutes it was back and everything seems to work fine.
I assumed it had ran out of space, and indeed df showed that /dev/md2 was 100% full (even though it says it's 9.8G in size and had 9.5G in it). I deleted some stuff and got that down to 50%. No new files in there though, I feel that's been full for a month or more...
I then had a look in /var/log/messages, to see what was last reported, and found this:
Apr 16 18:47:57 originals systemd[1]: Starting dnf makecache...
Apr 16 18:48:28 originals dnf[87659]: CentOS Stream 8 - AppStream 20 kB/s | 4.4 kB 00:00
Apr 16 18:48:28 originals dnf[87659]: CentOS Stream 8 - BaseOS 38 kB/s | 3.9 kB 00:00
Apr 16 18:48:28 originals dnf[87659]: CentOS Stream 8 - Extras 35 kB/s | 2.9 kB 00:00
Apr 16 18:48:28 originals dnf[87659]: CentOS Stream 8 - Extras common packages 19 kB/s | 3.0 kB 00:00
Apr 16 18:48:29 originals dnf[87659]: Extra Packages for Enterprise Linux 8 - x86_64 33 kB/s | 23 kB 00:00
Apr 16 18:48:30 originals dnf[87659]: Extra Packages for Enterprise Linux 8 - Next - 57 kB/s | 25 kB 00:00
Apr 16 18:48:30 originals dnf[87659]: Google Cloud SDK 3.8 kB/s | 1.4 kB 00:00
Apr 16 18:48:31 originals dnf[87659]: nginx mainline repo 160 kB/s | 2.9 kB 00:00
Apr 16 18:48:32 originals dnf[87659]: Metadata cache created.
Apr 16 18:48:32 originals systemd[1]: dnf-makecache.service: Succeeded.
Apr 16 18:48:32 originals systemd[1]: Started dnf makecache.
Apr 16 18:49:28 originals systemd[1]: dev-disk-by\x2duuid.device: Job dev-disk-by\x2duuid.device/start timed out.
Apr 16 18:49:28 originals systemd[1]: Timed out waiting for device dev-disk-by\x2duuid.device.
Apr 16 18:49:28 originals systemd[1]: Dependency failed for /dev/disk/by-uuid.
Apr 16 18:49:28 originals systemd[1]: dev-disk-by\x2duuid.swap: Job dev-disk-by\x2duuid.swap/start failed with result 'dependency'.
Apr 16 18:49:28 originals systemd[1]: dev-disk-by\x2duuid.device: Job dev-disk-by\x2duuid.device/start failed with result 'timeout'.
Apr 16 18:49:28 originals systemd[1]: Timed out waiting for device dev-disk-by\x2duuid.device.
Apr 16 18:49:28 originals systemd[1]: Dependency failed for /dev/disk/by-uuid.
Apr 16 18:49:28 originals systemd[1]: dev-disk-by\x2duuid.swap: Job dev-disk-by\x2duuid.swap/start failed with result 'dependency'.
Apr 16 18:49:28 originals systemd[1]: dev-disk-by\x2duuid.device: Job dev-disk-by\x2duuid.device/start failed with result 'timeout'.
Can anyone suggest what I might need to worry about? Or will it just be down to that /dev/md2 being full, even though that's been full for a while? Or does the above suggest I have an imminent hardware failure?
EDIT:
Looking again at messages I can see that these x2uuid errors occur relatively frequently, both before and since the reboot, and are maybe completely unrelated.
