Score:3

Unable to ssh over WAN

in flag

System:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:    20.04
Codename:   focal

Openssh version:

$ ssh -V
OpenSSH_8.2p1 Ubuntu-4ubuntu0.9, OpenSSL 1.1.1f  31 Mar 2020

Problem:

I'm trying to set up SSH so I can access it over WAN (i.e. Internet). It works no problem over LAN:

% ssh 10.81.1.222 -p 22           
Enter passphrase for key '/Users/userid/.ssh/id_rsa': 

However it fails when I try over WAN. I'm using a Linksys Velop mesh router. I have confirmed:

  • Port 22 on my router is forwarded to port 22 on my server @10.81.1.222
  • My external IP address using https://api.ipify.org/
  • ufw is inactive using "sudo ufw status"
  • server @10.81.1.222 has an Internet connection (I can ping cnn.com)

I have tried two different Internet providers: my Android phone as a hotspot and Xfinity. The latter times out (I've read others have had problems recently using ssh w/ Xfinity), the former leads to connection refused. My ISP is AT&T.

Xfinity:

% ssh -v my_userid@###.###.###.### -p 22 
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/my_userid/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to ###.###.###.### [###.###.###.###] port 22.
debug1: connect to address ###.###.###.### port 22: Operation timed out
ssh: connect to host ###.###.###.### port 22: Operation timed out

Android:

% ssh -v my_userid@###.###.###.### -p 22
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/my_userid/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to ###.###.###.### [###.###.###.###] port 22.
debug1: connect to address ###.###.###.### port 22: Connection refused
ssh: connect to host ###.###.###.### port 22: Connection refused

Note:

While connected to Android, my phone was still connected to my router, so this continued to work:

% ssh 10.81.1.222 -p 22

While connected to Xfinity and after disconnecting my phone from my router, pinging the external IP address failed:

% ping ###.###.###.###
PING ###.###.###.### (###.###.###.###): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

Any suggestions?

I purged openssh and reinstalled while troubleshooting, so I'm currently using the default sshd_config, but I've pasted it here for completness.

#       $OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

Include /etc/ssh/sshd_config.d/*.conf

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem       sftp    /usr/lib/openssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#       X11Forwarding no
#       AllowTcpForwarding no
#       PermitTTY no
#       ForceCommand cvs server
vidarlo avatar
om flag
What external IP does your router show? It's possible that you're behind [cgnat](https://en.wikipedia.org/wiki/Carrier-grade_NAT).
deiter62 avatar
in flag
Thanks for responding @vidarlo. I made a pretty embarrassing error. All fixed now. I posted the solution.
Score:1
in flag

Doh!

I had my Linksys router configured properly, but I completely forgot about my ISP's router. Once I configured that router to port forward to my Linksys router, things improved.

I started running into this problem:

kex_exchange_identification: read: Connection reset by peer

I'd forgotten that I'd set up entries in /etc/hosts.allow ages ago. I replaced them with:

sshd : ALL

All good now except my pride!

vidarlo avatar
om flag
Excellent :) Remember to mark your answer as accepted as it solved your question. We've all made similar stupid mistakes :)
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.