Change default path of the whole .ssh folder

I have a problem with my server and for some reason (which are not important here) i'm not able to use my default home folder (/home/$USER). At the moment i'm setting my $HOME everytime i login so that when i cd ~ the path is changed to /newhome/$USER.

Everything seems to work besides ssh. The problem is that ssh keep searching for /home/$USER/.ssh folder for the public/private key and known_host instead of using /newhome/$USER/.ssh:

> ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/home/$USER/.ssh/id_rsa): 
Could not create directory '/home/$USER/.ssh': No such file or directory

I already changed the AuthorizedKeysFile option in sshd_config file, i can login using keys but the problem persists. How can i tell ssh that the new default home folder is /newhome/$USER/.ssh?

This might not be what you are looking for, but you can move the contents of your entire ~/.ssh directory to some other directory, then delete ~/.ssh and create a symbolic link to your new directory called ~/.ssh

mkdir ~/alternative
mv ~/.ssh/* ~/alternative
rm -rf ~/.ssh
ln -s ~/alternative ~/.ssh

This integrates well with the gnome-keyring-ssh service which acts like a wrapper for ssh-agent and prompts you, with a graphical popup, for your SSH key passphrase the first time you use it whilst logged in.

If you simply use the IdentityFile field under a Host config in ~/.ssh/config to point to a different directory, then gnome-keyring-ssh seems so have some trouble finding your SSH key and you'll be forced to enter your passsphrase into the terminal every time the key is used. I haven't figured out why, but I think gnome-keyring-ssh just has it built-in to only properly look at the contents of ~/.ssh.

If anyone knows how to configure gnome-keyring-ssh to look at any other folder besides ~/.ssh, please let me know.