We have many virtualization hosts with VM disks residing typically on LVM. When VM is migrated across hosts it maybe required to do some work like rebuilding initramfs. We got used to mount VM disk to the host /mnt, mount proc-dev-sys inside mnt and chrooting into mnt.
kpartx -a /dev/kubrick/vm-pooi-arr
mount /dev/mapper/kubrick-vm--pooi--arr2 /mnt/
cd /mnt
mount -t proc /proc proc/
mount --rbind /sys sys/
mount --rbind /dev dev/
chroot /mnt /bin/bash
This worked fine till newer systems. Fedora Server 36/37 become unusable after umounting /mnt/sys /mnt/dev/ and /mnt/proc. Typically /mnt/dev is busy with [kdevtmpfs] process, but lazy umount do the work, but then all the system become totally unusable.
Daemons report:
Jan 24 17:50:25 kubrick virtqemud[1916]: Failed to open file '/sys/fs/cgroup/machine.slice/machine-qemu\x2d33\x2dexchange.scope/libvirt/cpu.stat': No such file or directory
Nothing can be restarted:
Jan 24 17:52:08 kubrick systemd[1]: virtqemud.service: Failed to create cgroup /system.slice/virtqemud.service: No such file or directory
/sys/fs/cgroup/ is empty, but /sys is still intact. Nobody can log in via ssh, even reboot will not be finished ever. I am not sure its an issue of Fedora, I think I saw it on latest Oracle Linux too.
What may cause cgroups disruption? Can it be avoided? Can it be fixed if once happened?
