Hostbased authentication fails on CentOS Stream release 9.
I trying to connect via ssh from COS8 to COS9. (Conversely way I can do it -COS9 to COS8).
client hostname: COS8.abc.lan
server hostname: COS9.abc.lan
client ssh_config:
Host *
HostbasedAuthentication yes
EnableSSHKeysign yes
Port 222
server sshd_config:
Port 222
ListenAddress 1.2.3.4
DenyUsers root
AllowUsers user1 user2
HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key
CASignatureAlgorithms rsa-sha2-512,rsa-sha2-256
LogLevel DEBUG3
PermitRootLogin no
AuthorizedKeysFile .ssh/authorized_keys
HostbasedAuthentication yes
IgnoreUserKnownHosts yes
IgnoreRhosts yes
KbdInteractiveAuthentication no
UsePAM yes
UseDNS yes
/etc/hosts on both machines:
1.2.3.3 COS8.abc.lan COS8 c8
1.2.3.4 COS9.abc.lan COS9 c9
/etc/ssh/shosts.equiv on both machines:
COS8.abc.lan
COS9.abc.lan
/etc/ssh/ssh_known_hosts2 on both machines has entries like this:
c8,COS8,COS8.abc.lan,1.2.3.3 ssh-rsa <public rsa C8 host key>
c9,COS9,COS9.abc.lan,1.2.3.4 ssh-rsa <public rsa C9 host key>
[c8]:222 ssh-rsa <public rsa C8 host key>
[COS8]:222 ssh-rsa <public rsa C8 host key>
[COS8.abc.lan]:222 ssh-rsa <public rsa C8 host key>
[1.2.3.3]:222 ssh-rsa <public rsa C8 host key>
[c9]:222 ssh-rsa <public rsa C9 host key>
[COS9]:222 ssh-rsa <public rsa C9 host key>
[COS9.abc.lan]:222 ssh-rsa <public rsa C9 host key>
[1.2.3.4]:222 ssh-rsa <public rsa C9 host key>
Permisions of above file are(on both machines):
-rw-r--r--. 1 root root 51242 Aug 30 22:50 /etc/ssh/ssh_known_hosts2
Permisions of /usr/libexec/openssh/ssh-keysign:
-r-xr-sr-x. 1 root ssh_keys 341272 Jul 20 12:18 /usr/libexec/openssh/ssh-keysign
Log from client c8 when trying to connect to c9 (ssh -vvv c9):
<I can't attach it because it has to many characters to this post could be created>
Client receive packet: type 51 from server:
debug1: userauth_hostbased: trying hostkey ssh-rsa SHA256:sH3z...
debug2: userauth_hostbased: chost COS8.abc.lan.
debug3: ssh_msg_send: type 2
debug3: ssh_msg_recv entering
debug3: ssh_keysign: [child] pid=49742, exec /usr/libexec/openssh/ssh-keysign
debug3: send packet: type 50
debug2: we sent a hostbased packet, wait for reply
debug3: receive packet: type 51
Thats mean that server does not authorized it for properly rsa key :(
So take a look on log on server side - c9 (journalctl -u sshd):
Starting OpenSSH server daemon...
debug3: already daemonized
debug3: oom_adjust_setup
Started OpenSSH server daemon.
debug1: Set /proc/self/oom_score_adj from 0 to -1000
debug2: fd 3 setting O_NONBLOCK
debug1: Bind to port 222 on 1.2.3.4.
Server listening on 1.2.3.4 port 222.
debug3: fd 4 is not O_NONBLOCK
debug1: Forked child 2737.
debug3: send_rexec_state: entering fd = 7 config len 3847
debug3: ssh_msg_send: type 0
debug3: send_rexec_state: done
debug3: oom_adjust_restore
debug1: Set /proc/self/oom_score_adj to 0
debug1: rexec start in 4 out 4 newsock 4 pipe 6 sock 7
debug1: inetd sockets after dupping: 4, 4
Connection from 1.2.3.3 port 42806 on 1.2.3.4 port 222 rdomain ""
debug1: Local version string SSH-2.0-OpenSSH_8.7
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 4 setting O_NONBLOCK
debug3: ssh_sandbox_init: preparing seccomp filter sandbox
debug2: Network child is on pid 2738
debug3: preauth child monitor started
debug1: SELinux support enabled [preauth]
debug1: ssh_selinux_change_context: setting context from 'system_u:system_r:sshd_t:s0-s0:c0.c1023' to 'system_u:system_r:sshd_net_t:s0-s0:c0.c1023' [preauth]
debug3: privsep user:group 74:74 [preauth]
debug1: permanently_set_uid: 74/74 [preauth]
debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth]
debug3: ssh_sandbox_child: attaching seccomp filter program [preauth]
debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth]
debug3: send packet: type 20 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
debug3: receive packet: type 20 [preauth]
debug1: SSH2_MSG_KEXINIT received [preauth]
debug2: local server KEXINIT proposal [preauth]
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1 [preauth]
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth]
debug2: ciphers ctos: [email protected],[email protected],aes256-ctr,[email protected],aes128-ctr [preauth]
debug2: ciphers stoc: [email protected],[email protected],aes256-ctr,[email protected],aes128-ctr [preauth]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 [preauth]
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 [preauth]
debug2: compression ctos: none,[email protected] [preauth]
debug2: compression stoc: none,[email protected] [preauth]
debug2: languages ctos: [preauth]
debug2: languages stoc: [preauth]
debug2: first_kex_follows 0 [preauth]
debug2: reserved 0 [preauth]
debug2: peer client KEXINIT proposal [preauth]
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c [preauth]
debug2: host key algorithms: [email protected],[email protected],[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519 [preauth]
debug2: ciphers ctos: [email protected],[email protected],aes256-ctr,aes256-cbc,[email protected],aes128-ctr,aes128-cbc [preauth]
debug2: ciphers stoc: [email protected],[email protected],aes256-ctr,aes256-cbc,[email protected],aes128-ctr,aes128-cbc [preauth]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 [preauth]
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 [preauth]
debug2: compression ctos: none,[email protected],zlib [preauth]
debug2: compression stoc: none,[email protected],zlib [preauth]
debug2: languages ctos: [preauth]
debug2: languages stoc: [preauth]
debug2: first_kex_follows 0 [preauth]
debug2: reserved 0 [preauth]
debug1: kex: algorithm: curve25519-sha256 [preauth]
debug1: kex: host key algorithm: rsa-sha2-512 [preauth]
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none [preauth]
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none [preauth]
debug1: kex: curve25519-sha256 need=32 dh_need=32 [preauth]
debug3: mm_request_send: entering, type 120 [preauth]
debug3: mm_request_receive_expect: entering, type 121 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 120
debug3: mm_request_send: entering, type 121
debug1: kex: curve25519-sha256 need=32 dh_need=32 [preauth]
debug3: mm_request_send: entering, type 120 [preauth]
debug3: mm_request_receive_expect: entering, type 121 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 120
debug3: mm_request_send: entering, type 121
debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
debug3: receive packet: type 30 [preauth]
debug1: SSH2_MSG_KEX_ECDH_INIT received [preauth]
debug3: mm_sshkey_sign: entering [preauth]
debug3: mm_request_send: entering, type 6 [preauth]
debug3: mm_sshkey_sign: waiting for MONITOR_ANS_SIGN [preauth]
debug3: mm_request_receive_expect: entering, type 7 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 6
debug3: mm_answer_sign: entering
debug3: mm_answer_sign: rsa-sha2-512 (effective: rsa-sha2-512) KEX signature len=404
debug3: mm_request_send: entering, type 7
debug2: monitor_read: 6 used once, disabling now
debug3: send packet: type 31 [preauth]
debug3: send packet: type 21 [preauth]
debug2: set_newkeys: mode 1 [preauth]
debug1: rekey out after 4294967296 blocks [preauth]
debug1: SSH2_MSG_NEWKEYS sent [preauth]
debug1: Sending SSH2_MSG_EXT_INFO [preauth]
debug3: send packet: type 7 [preauth]
debug1: expecting SSH2_MSG_NEWKEYS [preauth]
debug3: receive packet: type 21 [preauth]
debug1: SSH2_MSG_NEWKEYS received [preauth]
debug2: set_newkeys: mode 0 [preauth]
debug1: rekey in after 4294967296 blocks [preauth]
debug1: KEX done [preauth]
debug3: receive packet: type 5 [preauth]
debug3: send packet: type 6 [preauth]
debug3: receive packet: type 50 [preauth]
debug1: userauth-request for user user1 service ssh-connection method none [preauth]
debug1: attempt 0 failures 0 [preauth]
debug3: mm_getpwnamallow: entering [preauth]
debug3: mm_request_send: entering, type 8 [preauth]
debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth]
debug3: mm_request_receive_expect: entering, type 9 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 8
debug3: mm_answer_pwnamallow: entering
debug3: Trying to reverse map address 1.2.3.3.
debug2: parse_server_config_depth: config reprocess config len 3847
debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/50-redhat.conf len 720
debug2: parse_server_config_depth: config /etc/crypto-policies/back-ends/opensshserver.config len 1982
debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1
debug3: mm_request_send: entering, type 9
debug2: monitor_read: 8 used once, disabling now
debug2: input_userauth_request: setting up authctxt for user1 [preauth]
debug3: mm_start_pam entering [preauth]
debug3: mm_request_send: entering, type 100 [preauth]
debug3: mm_inform_authserv: entering [preauth]
debug3: mm_request_send: entering, type 4 [preauth]
debug3: mm_inform_authrole: entering [preauth]
debug3: mm_request_send: entering, type 80 [preauth]
debug2: input_userauth_request: try method none [preauth]
debug3: user_specific_delay: user specific delay 0.000ms [preauth]
debug3: ensure_minimum_time_since: elapsed 3.384ms, delaying 3.111ms (requested 6.495ms) [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 100
debug1: PAM: initializing for "user1"
debug1: PAM: setting PAM_RHOST to "COS8.abc.lan"
debug1: PAM: setting PAM_TTY to "ssh"
debug2: monitor_read: 100 used once, disabling now
debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password,hostbased" [preauth]
debug3: send packet: type 51 [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 4
debug3: mm_answer_authserv: service=ssh-connection, style=
debug2: monitor_read: 4 used once, disabling now
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 80
debug3: mm_answer_authrole: role=
debug2: monitor_read: 80 used once, disabling now
debug3: receive packet: type 50 [preauth]
debug1: userauth-request for user user1 service ssh-connection method hostbased [preauth]
debug1: attempt 1 failures 0 [preauth]
debug2: input_userauth_request: try method hostbased [preauth]
debug1: userauth_hostbased: cuser user1 chost COS8.abc.lan. pkalg ecdsa-sha2-nistp256 slen 101 [preauth]
debug3: mm_key_allowed: entering [preauth]
debug3: mm_request_send: entering, type 22 [preauth]
debug3: mm_key_allowed: waiting for MONITOR_ANS_KEYALLOWED [preauth]
debug3: mm_request_receive_expect: entering, type 23 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 22
debug3: mm_answer_keyallowed: entering
debug2: hostbased_key_allowed: chost COS8.abc.lan. resolvedname COS8.abc.lan ipaddr 1.2.3.3
debug2: stripping trailing dot from chost COS8.abc.lan.
debug2: auth_rhosts2: clientuser user1 hostname COS8.abc.lan ipaddr 1.2.3.3
debug1: temporarily_use_uid: 1000/1000 (e=0/0)
debug1: restore_uid: 0/0
debug1: fd 5 clearing O_NONBLOCK
debug2: hostbased_key_allowed: access allowed by auth_rhosts2
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: check_key_in_hostfiles: key for host COS8.abc.lan not found
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: Permission denied
debug1: check_key_in_hostfiles: key for host COS8.abc.lan not found
debug3: mm_answer_keyallowed: hostbased authentication test: ECDSA key is not allowed
Failed hostbased for user1 from 1.2.3.3 port 42806 ssh2: ECDSA SHA256:GDsQ..., client user "user1", client host "COS8.abc.lan"
debug3: mm_request_send: entering, type 23
debug2: userauth_hostbased: authenticated 0 [preauth]
debug3: user_specific_delay: user specific delay 0.000ms [preauth]
debug3: ensure_minimum_time_since: elapsed 5.586ms, delaying 0.909ms (requested 6.495ms) [preauth]
debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password,hostbased" [preauth]
debug3: send packet: type 51 [preauth]
debug3: receive packet: type 50 [preauth]
debug1: userauth-request for user user1 service ssh-connection method hostbased [preauth]
debug1: attempt 2 failures 1 [preauth]
debug2: input_userauth_request: try method hostbased [preauth]
debug1: userauth_hostbased: cuser user1 chost COS8.abc.lan. pkalg ssh-ed25519 slen 83 [preauth]
debug3: mm_key_allowed: entering [preauth]
debug3: mm_request_send: entering, type 22 [preauth]
debug3: mm_key_allowed: waiting for MONITOR_ANS_KEYALLOWED [preauth]
debug3: mm_request_receive_expect: entering, type 23 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 22
debug3: mm_answer_keyallowed: entering
debug2: hostbased_key_allowed: chost COS8.abc.lan. resolvedname COS8.abc.lan ipaddr 1.2.3.3
debug2: stripping trailing dot from chost COS8.abc.lan.
debug2: auth_rhosts2: clientuser user1 hostname COS8.abc.lan ipaddr 1.2.3.3
debug1: temporarily_use_uid: 1000/1000 (e=0/0)
debug1: restore_uid: 0/0
debug1: fd 5 clearing O_NONBLOCK
debug2: hostbased_key_allowed: access allowed by auth_rhosts2
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: check_key_in_hostfiles: key for host COS8.abc.lan not found
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: Permission denied
debug1: check_key_in_hostfiles: key for host COS8.abc.lan not found
debug3: mm_answer_keyallowed: hostbased authentication test: ED25519 key is not allowed
Failed hostbased for user1 from 1.2.3.3 port 42806 ssh2: ED25519 SHA256:f2og..., client user "user1", client host "COS8.abc.lan"
debug3: mm_request_send: entering, type 23
debug2: userauth_hostbased: authenticated 0 [preauth]
debug3: user_specific_delay: user specific delay 0.000ms [preauth]
debug3: ensure_minimum_time_since: elapsed 1.425ms, delaying 5.070ms (requested 6.495ms) [preauth]
debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password,hostbased" [preauth]
debug3: send packet: type 51 [preauth]
debug3: receive packet: type 50 [preauth]
debug1: userauth-request for user user1 service ssh-connection method hostbased [preauth]
debug1: attempt 3 failures 2 [preauth]
debug2: input_userauth_request: try method hostbased [preauth]
debug1: userauth_hostbased: cuser user1 chost COS8.abc.lan. pkalg ssh-rsa slen 399 [preauth]
debug3: mm_key_allowed: entering [preauth]
debug3: mm_request_send: entering, type 22 [preauth]
debug3: mm_key_allowed: waiting for MONITOR_ANS_KEYALLOWED [preauth]
debug3: mm_request_receive_expect: entering, type 23 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 22
debug3: mm_answer_keyallowed: entering
debug2: hostbased_key_allowed: chost COS8.abc.lan. resolvedname COS8.abc.lan ipaddr 1.2.3.3
debug2: stripping trailing dot from chost COS8.abc.lan.
debug2: auth_rhosts2: clientuser user1 hostname COS8.abc.lan ipaddr 1.2.3.3
debug1: temporarily_use_uid: 1000/1000 (e=0/0)
debug1: restore_uid: 0/0
debug1: fd 5 clearing O_NONBLOCK
debug2: hostbased_key_allowed: access allowed by auth_rhosts2
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: check_key_in_hostfiles: key for host COS8.abc.lan not found
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: Permission denied
debug1: check_key_in_hostfiles: key for host COS8.abc.lan not found
debug3: mm_answer_keyallowed: hostbased authentication test: RSA key is not allowed
Failed hostbased for user1 from 1.2.3.3 port 42806 ssh2: RSA SHA256:sH3z..., client user "user1", client host "COS8.abc.lan"
debug3: mm_request_send: entering, type 23
debug2: userauth_hostbased: authenticated 0 [preauth]
debug3: user_specific_delay: user specific delay 0.000ms [preauth]
debug3: ensure_minimum_time_since: elapsed 1.437ms, delaying 5.058ms (requested 6.495ms) [preauth]
debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password,hostbased" [preauth]
debug3: send packet: type 51 [preauth]
debug3: mm_request_send: entering, type 122 [preauth]
debug3: mm_request_receive_expect: entering, type 123 [preauth]
debug3: mm_request_receive: entering [preauth]
debug3: mm_request_receive: entering
debug3: monitor_read: checking request 122
debug3: mm_request_send: entering, type 123
Connection closed by authenticating user user1 1.2.3.3 port 42806 [preauth]
debug1: do_cleanup [preauth]
debug3: PAM: sshpam_thread_cleanup entering [preauth]
debug1: monitor_read_log: child log fd closed
debug3: mm_request_receive: entering
debug1: do_cleanup
debug1: PAM: cleanup
debug3: PAM: sshpam_thread_cleanup entering
debug1: Killing privsep child 2738
What worry me are that two things:
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: Permission denied
debug3: mm_answer_keyallowed: hostbased authentication test: RSA key is not allowed
AD 1) Why permisions are denied? To who they are denied? Permisions to this file are:
-rw-r--r--. 1 root root
so everyone can read this file !!
AD 2) Why RSA key is not allowed? When I try with ED25519 key, information was the same for it:
debug2: hostbased_key_allowed: access allowed by auth_rhosts2
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: check_key_in_hostfiles: key for host headnode2.pbs.lan not found
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: Permission denied
debug1: check_key_in_hostfiles: key for host headnode2.pbs.lan not found
debug3: mm_answer_keyallowed: hostbased authentication test: ED25519 key is not allowed
Before do this I was uncommented those two lines in sshd_config file and restarted sshd:
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key
Can anyone help with those?
