Score:0

What is the host key (the one from ssh connection) and how is it different from public-private key pair?

br flag

The situation is that I've had a VPS created previously. It was all set up, private-public key authentication, root login turned off, password login turned off. Everything was set up.

Then this server gets destroyed and a new server gets spun-off.

So I'm using ssh -v root@new_server_ip_number to log into this newly installed linux instance and this is what I get:

PS C:\Users\roeslermichal> ssh -v [email protected]
OpenSSH_for_Windows_8.6p1, LibreSSL 3.4.3
debug1: Reading configuration data C:\\Users\\roeslermichal/.ssh/config
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to 10.32.81.216 [10.32.81.216] port 22.
debug1: Connection established.
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_rsa type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_rsa-cert type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_dsa type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_dsa-cert type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ecdsa type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ecdsa-cert type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ecdsa_sk type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ed25519 type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ed25519-cert type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ed25519_sk type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_ed25519_sk-cert type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_xmss type -1
debug1: identity file C:\\Users\\roeslermichal/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 10.32.81.216:22 as 'root'
debug1: load_hostkeys: fopen C:\\Users\\roeslermichal/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:5OrjMYiYdmoRTDgjsmBfOXun/4FpiClOU6L21gBDPSk
debug1: load_hostkeys: fopen C:\\Users\\roeslermichal/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:5OrjMYiYdmoRTDgjsmBfOXun/4FpiClOU6L21gBDPSk.
Please contact your system administrator.
Add correct host key in C:\\Users\\roeslermichal/.ssh/known_hosts to get rid of this message.
Offending RSA key in C:\\Users\\roeslermichal/.ssh/known_hosts:15
Host key for 10.32.81.216 has changed and you have requested strict checking.
Host key verification failed.

What is this SHA256:5OrjMYiYdmoRTDgjsmBfOXun/4FpiClOU6L21gBDPSk. line? What does it mean?
Because clearly this SHA256:5OrjMYiYdmoRTDgjsmBfOXun/4FpiClOU6L21gBDPSk. number/id is not the same number that identifies the linux server in my Windows known_hosts file.

I'm using Windows laptop and PowerShell to log into this server.
There's a C:\Users\roeslermichal\.ssh\known_hosts file on this Windows machine and I've been expecting that some IDs won't match, because the old server got destroyed and the new one got created. I've already deleted the 10.32.81.216 ssh-rsa key and I've replaced it with this 10.32.81.216 ssh-rsa key of the newly installed linux server.
But the ssh-client won't let me in.
This is how my current :\Users\roeslermichal\.ssh\known_hosts file looks like:

10.32.81.216 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGh8fEmrCov7TLbiKgGasUV3fxbrKmh4Ai/RWixt41Fl
10.32.81.216 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCr43mfkweJAaHQ4kw88b0y5OShnQl91jR1eoUIcnaMRvBEi3X7McVuA+cB+MWk4Rj9EX2hnq6tyB+26weQX0GXWf95CL/yqX5p39b+j8c43CR9/3gHbU5aV+exGBbj2rEL4JgmQD58fHHEsL1r6EMcpTUgY8JqfG0F52XUJrF7KdpxlW4vtgOaHdqooBMHuMi+bR7LRq/moAHLv3svB5PPhIfSbM5CW/Eke4H4qiAwKCVUjyXxKCoKkYVDyfQur+nBMxJssUHy03385hxV0gKo8WGQKlSNvI3B1vP85ij5zCYViYUfs05lXPkpsUqosGqHDOJhPnVRM4OacMQVkj2e0MKHs/cXA1GneBiY99tPMaEL2qZ0UJoaYcnG0krc0owKE6Ufx+84VVqLG7hJHPnNRI3UrFjG/C7lAzAogz5eDiYoQvkko7mLuwRob27fIB39oH2cbH4a4DCcIDekS0WwCPeA+uwaHrmhKJluqP8r7qvDluWax3cVzDGojD7I6cU=
10.32.81.216 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCMktfR/tBD8GYWRWpo8DsoIPPxos+Rt/C1Is04S0Dglm6UbQqQQUW9m9GfDWHZn3j37ZWPGeUwTcWEojKi70yk=

10.32.81.218 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFBuju3Gav0s6Uj8XFQToa/qU7gxsxvKqtUCctWaC4FC
10.32.81.218 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6OCnBNeCfiLcYo7FAmopNBxWS5No+Locw+dxujELxhXn/zAEEnsMv+fZYP8JT8Jj+bYFX1jVAxBubqaz7swK3GCYkkL4C/dI2p7MV0E0ogznbZEZS0GHU3wA69R7s4F56oR3ZeCIas+gfe3mckB4i9UtZMy2IsGSVl974wletCXfdXxhkyRzHlgovoCnAYu9qOS/X2X2yuUNKKfL3VGQNkAih/Hjqh7Iwi36sLS8+WB/sYOk5cxJfycWewTEl1Wt5fB5bbc7Fu0Wmjn2IpMHspoR6YEw2lK/GuFIFjcVoHJ8+7JAuY9BnUdyuAbHLZ8vgrymcGw/ZP8GIhgRq1nOseAQrOzZMFtcGCS953a+L5gP9shX2ZwF/MS7h8+EYPxMNFZP6DbU++c4ZmOlb0lPkUJDhTnSbOoDZA+bfDl5jBlKtfF2V7n+V9Dwuwwbsp/qJyULIeMAdCrpjPhmKhnQASloZsEN5LLjh2gVN+YM7jACHe6ZyFD4/gpEE6N6MUG8=
10.32.81.218 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBonnCuOeQpc7CSRzbps8sLnPYMphNrfqs9h7Hz5I+Ml8QxPBUnlNw749EzqC29KFtyB8XE2SnbOK/CuUnghj5E=

But I don't really know what are those host keys, because I haven't created any keys on this new linux server. Login approach as root was the first action I've made regarding this newly created linux server. And there are already some host_keys on this server.??? And these are not private-public ssh keys, because I haven't created them yet, so what are those keys, that got generated and identify my new linux server in the Windows known_hosts file.

I've read this thread carefully few times, but don't quite understand the answers given there and why they work. Even more I don't understand why I can't log into my newly created linux server, although I've replaced the old sever rsa host key with a new server rsa host key.

PS C:\Users\roeslermichal> ssh-keyscan -t rsa 10.32.81.216
# 10.32.81.216:22 SSH-2.0-OpenSSH_8.0
10.32.81.216 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCr43mfkweJAaHQ4kw88b0y5OShnQl91jR1eoUIcnaMRvBEi3X7McVuA+cB+MWk4Rj9EX2hnq6tyB+26weQX0GXWf95CL/yqX5p39b+j8c43CR9/3gHbU5aV+exGBbj2rEL4JgmQD58fHHEsL1r6EMcpTUgY8JqfG0F52XUJrF7KdpxlW4vtgOaHdqooBMHuMi+bR7LRq/moAHLv3svB5PPhIfSbM5CW/Eke4H4qiAwKCVUjyXxKCoKkYVDyfQur+nBMxJssUHy03385hxV0gKo8WGQKlSNvI3B1vP85ij5zCYViYUfs05lXPkpsUqosGqHDOJhPnVRM4OacMQVkj2e0MKHs/cXA1GneBiY99tPMaEL2qZ0UJoaYcnG0krc0owKE6Ufx+84VVqLG7hJHPnNRI3UrFjG/C7lAzAogz5eDiYoQvkko7mLuwRob27fIB39oH2cbH4a4DCcIDekS0WwCPeA+uwaHrmhKJluqP8r7qvDluWax3cVzDGojD7I6cU=

Although I haven't replaced the 10.32.81.216 ssh-ed25519 nor 10.32.81.216 ecdsa-sha2-nistp256 key. Can this be the reason I'm unable to login?

Jaromanda X avatar
ru flag
`What does it mean?` it's the server host key - `is not the same number that identifies the linux server in my Windows known_hosts file` correct, because it's a key that is generated when the server is first created - simply remove line 15 in `C:\\Users\\roeslermichal/.ssh/known_hosts`
dave_thompson_085 avatar
jp flag
@Jaromanda+ Actually the "what does it mean" value is the _fingerprint_ of the host key, not the actual host key; that's why it says `SHA256:` at the beginning. However since you (OP) didn't update the ed25519 key in known_hosts, that key is still the one for the old server instance and now wrong and harmful, so yes you should replace it. And you probably should either replace or remove the ecdsa key to avoid possible future problems or confusion, although it's not your current problem.
Jaromanda X avatar
ru flag
*the fingerprint of the host key* correct, but not important to the understanding of the issue
michal roesler avatar
br flag
I forgot to mention it in the question, but I've tampered with a `~/.ssh/known_hosts` file a little. I've deleted some lines referring to the other online servers I got. They were earlier in the file, before 10.32.81.216 and 10.32.81.218 entries. That's why there's `Offending RSA key in C:\\Users\\roeslermichal/.ssh/known_hosts:15' line 15, while there isn't that many lines in the `known_hosts` contents presented in the question.
michal roesler avatar
br flag
Although I've accepted the answer, because the deleting server host keys using `ssh-keygen -R 10.32.81.216` worked fine (I was able to log in), there is a problem I don't understand. Before I've deleted those host keys, I've tried to REPLACE the 10.32.81.216 `ecdsa-sha2-nistp256 key` using the `ssh-keyscan -t rsa,ecdsa 10.32.81.216` output and manually REPLACED the value in `known_hosts` file and it didn't work. How can it be, that REPLACING host keys in `~/.ssh/known_hosts` file doesn't work? Is this file free to edit using copy/paste method or not?
Nikita Kipriyanov avatar
za flag
This file is free to edit. Probably you put in wrong key, or put it in a wrong way. It is a perfectly supported way to [build a known_hosts file](https://serverfault.com/a/316100/325117) using the output of ssh-keyscan.
michal roesler avatar
br flag
@NikitaKipriyanov If what you write is correct, can I perform this kind of experiment? First, delete the whole contents of the `~/.ssh/known_hosts` file, but keep the file itself. Second, run the `ssh-keyscan -t ecdsa 10.32.81.216` in powershell and copy the output to the clipboard. Third, Paste the clipboard contents to the empty file and save it. Should this work?
Nikita Kipriyanov avatar
za flag
There are subtle things like UTF-8 BOM, line endings etc, which Windows editors might do incorrectly (Notepad is notorious for that) and OpenSSH tools might be sensitive to that. Probably *this* was the problem with your manual editing of the file. I believe shell redirection (like `ssh-keyscan host > known_hosts`) instead of manual copy-paste can avoid at least some of such problems.
Score:1
za flag

There is a mutual authentication in SSH.

First, the client authenticates the server is indeed the one it wanted to connect. For that, it remembers the public part of the host keypair in the ~/.ssh/known_hosts file. It learns that either during the first connection (this is exactly the part where it asks you to type "yes"), or may learn from DNS if it contains the SSHFP record for the host and the zone is DNSSEC-protected. If client finds out that the server is presenting the wrong key, it will normally refuse to connect claiming an ongoing MitM attack.

This is what SSH host key pair is for. Roughly speaking it is the SSH version of the PKI infrastructure albeit not a CA-based one (or it uses DNSSEC to implement a CA-like trust chain); it's like HTTPS certificate/key pair (with the purpose "web server authentication") and serves the same purpose. It is the asymmetric ("public-private") key pair that belongs to a server.

Secondly, the server authenticates the client is really who it claims to be. For that, it can use username/password pair or whatever complex chat-based authentication, or the asymmetric key pair stored on the server in users ~/.ssh/authorized_keys. This time, the key pair belongs to an user. Again, traditional CA-based PKI has an "analogue", client certificates (with the purpose "web client authentication").


Well, do you see this line in your output, Someone could be eavesdropping on you right now (man-in-the-middle attack)!? This is how it reports the MitM attack it designed to prevent. Probably it is overly cautious, but this is your security.

If you're absolutely sure there is no attack and the new fingerprint is the correct one, simply remove the offending line from your client's known_hosts file. You can follow the
@JaromandaX advice in the comment or you can remove all offending records with something like

ssh-keygen -R 10.32.81.216

Then you'll have to agree by typing literal "yes" to the question about you are sure, or by using a ssh-keyscan utility as described in the other answer. Notice, while this way of building the file avoids the interactive warning about MitM, it is still susceptible to the attack for the same reason, which is also noted in the manual page of ssh-keyscan (and also mentioned many times in comments to the answers under the link).

dave_thompson_085 avatar
jp flag
No, `known_hosts` remembers the actual host public key, as a base64 blob plus some metadata. But if the received key differs from the remembered value -- as when the server is recreated -- `ssh` _displays_ the fingerprint. And SSHFP in DNS (not KEY) can contain that fingerprint, hence the name. Also you can run `ssh-keygen -R hostid` without `-f anything` and it automatically uses the same file as `ssh` does.
michal roesler avatar
br flag
So what exactly is this line for `The fingerprint for the ED25519 key sent by the remote host is SHA256:5OrjMYiYdmoRTDgjsmBfOXun/4FpiClOU6L21gBDPSk.` - how am I supposed to use it? This SHA256 value is not of the same type, as the values in the `C:\Users\roeslermichal\.ssh\known_hosts` file, so how am I supposed 2 use it if it's not the right fingerprint/id to be copied into the `C:\Users\roeslermichal\.ssh\known_hosts` file in my Windows laptop?
Nikita Kipriyanov avatar
za flag
@dave_thompson_085 thank you for corrections! I was too lazy to look for the exact DNS RR type. @michalroesler Ideally you must use some prior secure channel to verify the key presented over the network is correct and there is no ongoing MitM attack. You are supposed to go to that host, find out the fingerprint of the key (by using `ssh-keygen`) and compare whatever you see over the network and at the terminal. In many cases, however, you "just assume" there is no attack right now (heuristics: you control the network, attack is unlikely and so on) and trust whatever you see on the screen.
michal roesler avatar
br flag
Coming back this question: `https://serverfault.com/questions/321167/add-correct-host-key-in-known-hosts-multiple-ssh-host-keys-per-hostname` - the accepted answer - Is there any way to edit this command: `$ ssh-keyscan -t rsa 10.32.81.216` in such a way, that it provides a key, I can copy into the `~/.ssh/known_hosts` file and log in passing host key identification? It looks like the client doesn't look for this host's `rsa` key, because I've updated it, and it doesn't matter. The client checks and is offended by the `ecdsa-sha2-nistp256` key.
Nikita Kipriyanov avatar
za flag
Yes, use not "-t rsa" but the type you are interested in. You can also omit -t parameter, it will receive rsa, ecdsa and ed25519 keys (see `man ssh-keyscan` for details). However, this is no more secure than just typing "yes" during first connection, because the same implication holds: if there's ongoing MitM you'll see the key from attacker host, not from your target host (there is a warning about this in the man page).
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.