I followed similar steps inspired from Octal.
First, I ran
sudo apt update
sudo apt remove '^nvidia'
sudo apt autoremove
In the above command, please make sure to run "sudo apt update" before removing nvidia packages. Without that update command, this issue won't be fixed.
Then, I ran
sudo apt-get purge 'nvidia*'
sudo apt autoremove
Then, I rebooted (this step is very very important. Don't immediately install the driver without rebooting).
Afterwards, I ran
sudo apt install nvidia-driver-460
and, reboot, and nvidia-smi should no longer cause any error.
Also, I agree with Octal, I think there is a compatibility issue between nvidia-driver-470, Ubuntu 20.04, and kernel 5.11.xxx because after I ran the steps that I have mentioned, I tried to install nvidia-driver-470 and nvidia-smi error returned.
EDIT:
after the step above, I faced the same issue when I booted up my PC in the next day.
The solution to this issue is to uninstall the bbswitch-dkms package from your OS.
Then you need to reboot.
If after rebooting, your login menu somehow not showing, try to plug out your HDMI cable, and then, the login menu should show up.
After you are already logged in, plug in the HDMI cable again.