I'm currently running a deep learning server with Ubuntu 22.04 LTS.
After a reboot, it keeps running into kernel panic with the error messages '/sbin/init not found' and '/etc/init permission denied'. Recovery mode does not help at all.
Based on a quick search, I found that this problem usually occurs after a partial update corrupting the initrd image. Following the typical approach to regenerate the initrd image, I booted into the Ubuntu 22.04 LTS live 'Try Ubuntu' using a USB stick. I checked the partitions using 'sudo parted -l', which showed that the root partition '/' was at '/dev/nvme0n1p3' and that the boot partition '/boot' was at '/dev/nvme0n1p2'.
On the live environment, I mounted these two partitions at '/mnt' and '/mnt/boot', and 'mount --bind'ed some directories (/dev, /dev/pts, /proc, and /sys). Then I 'chroot'ed into the '/mnt' directory. The networking was working in both the live environment and inside chroot, after supplying the proper static IP configurations via the GUI network manager. I tried various commands to regenerate the initrd image, but none of them worked.
To explain it in detail, 'update-initramfs -u' initially failed at 'fuse' step. So, I reinstalled fuse via apt-get install --reinstall kmod, which fixed it. However, then, I started to see that 'udev' failed (returned 1). There was no other error messages that yielded some clue. This was not fixed by apt-get install --reinstall.
I tried other commands as below, but every single one resulted in the same udev error:
- apt-get install --reinstall init
- apt-get install --reinstall kmod
- apt-get install --reinstall initramfs-tools
- Copying the contents of /boot into a temp folder and running update-initramfs in the temp folder
Here are some other things that I have tried, but none of which worked:
- Checking the permission of /etc/init, which was 755
- Creating a symbolic link from /sbin/init into /lib/systemd/systemd with proper permissions (755)
- The weird thing was that even after creating /sbin/init as a symbolic link, the kernel panic message still insisted /sbin/init did not exist
Any help would be appreciated. Thanks.