Let's assume SSH, other protocols would be different. Users provide you a SSH public key. (You do not need to see their private key.) Configure this as a credential to a personal, non privilaged (not root, not sudo) user.
How to implement this varies greatly depending on the user identity system available to your organization. Which might not be a service of your cloud provider, you can auth against whatever you like.
A manual solution could be you creating local users on the server and uploading their ~/.ssh/authorized_keys
. Tedious to maintain, users do not have a self service option.
Google Cloud has OS Login where you can grant any Google account (personal or managed) access to instances with their personal ssh key. Users granted the roles/compute.osLogin
role are not privilaged, so they will not have sudo. Probably only makes sense if your organization already grants Google accounts, a Workspace domain or Cloud Identity.
Or you might have an identity system that is not from your cloud provider. FreeIPA for example can store ssh keys, and integrate those with ssh auth on hosts. Again requires a bit of work to deploy a identity system if it does not exist yet.