Score:2

How to recover from a corrupted glibc on Ubuntu 20.04 server

us flag

Given a second chance I would double check it, but I accidentally run sudo ln -sf something /usr/lib/x86_64-linux-gnu/libc-2.31.so. And there is not another copy of glibc because this is the real shared object instead of a symlink.

So how can I recover from it? I think I may have to get another copy glibc, boot the machine from a usb device, and copy the clean libc-2.31.so to /usr/lib/x86_64-linux-gnu.

What is the correct way to do it? Do I also have to rebuild the symlinks libc.so.6 under /lib/x86_64-linux-gnu and /usr/lib/x86_64-linux-gnu?

Score:1
zw flag

Do not create symlinks in the folders which are controlled by APT!!!
You will break the system and/or create a nightmare for its future maintenance!!!

To recover the situation reinstall the libc6 package by

sudo apt-get update
sudo apt-get install --reinstall libc6 

and do not touch symlinks next time.
If you want to play with the system - do this in /usr/local/bin which is outside of APT control.


Note: on the fresh minimal 20.04 LTS there is only one file named libc-2.31.so - see

# find / -name libc-2.31.so
/usr/lib/x86_64-linux-gnu/libc-2.31.so

and one named libc.so.6:

 # find / -name libc.so.6
 /usr/lib/x86_64-linux-gnu/libc.so.6

while they are symlinked as follows (central part is replaced by (...)):

 # ls -al /usr/lib/x86_64-linux-gnu/libc-2.31.so /usr/lib/x86_64-linux-gnu/libc.so.6
 -rwxr-xr-x (...) /usr/lib/x86_64-linux-gnu/libc-2.31.so
 lrwxrwxrwx (...) /usr/lib/x86_64-linux-gnu/libc.so.6 -> libc-2.31.so

Both are provided by libc6 deb-package.

Fengkai Sun avatar
us flag
Sorry I did not make it clear, but the server is currently down and cannot boot again because it says it cannot find `libc.so.6`. If I run the command in the bootable USB, it does not seem to install glibc on the original disk of the server, right?
N0rbert avatar
zw flag
Then you have to boot using live-USB, mount rootfs of the server filesystem, chroot to it and reinstall the `libc6` package there. Or copy both `/usr/lib/x86_64-linux-gnu/libc-2.31.so` and `/usr/lib/x86_64-linux-gnu/libc-2.31.so` from live system to server fs, then reboot to the OS and reinstall the latest `libc6` deb-package.
Fengkai Sun avatar
us flag
Thanks for your fast clarification! I run `find / -name libc-2.31.so` on a fresh live system, and get the same result as you. But when I run `ls /lib/x86_64-linux-gnu/ | grep libc`, there is still `libc-2.31` and `libc.so.6`...
N0rbert avatar
zw flag
I had a typo in previous comment. Thanks! I edited an answer to clarify this moment about symlinks between libc-2.31 and libc.so.6. I think it is now complete and you can accept it.
Fengkai Sun avatar
us flag
Thank you very much for the detailed advice! I'm still on my way to touch the server and would update the question if things don't work well.
N0rbert avatar
zw flag
I hope all will be going well. Be careful next time :)
michael avatar
us flag
fwiw, on my 22.04 ubuntu, my `libresolv.so` (a file also installed by `libc6*.deb`) recently somehow became corrupted (result: couldn't boot, *but* I *could* boot into recovery mode, but no internet). So to fix, I downloaded the deb package from a "live USB" to local disk, and then rebooted into "recovery" mode, and re-installed the package via `dpkg -i ./path/to/libc6_2.35-0ubuntu3.1_amd64.deb` (make sure to get correct pkg for your release). This _might_ also work in the situation described here, _assuming_ you can boot into recovery mode https://wiki.ubuntu.com/RecoveryMode
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.