Score:2

SSH Connection Closed after changing sources.list

ru flag

working on a Ubuntu 19.04 Digital Ocean server. After updating sources.list to focal, so I could update repositories and install certbot, I found my self locked out of the server with the message Connection refused.
Managed to enter through Digital Ocean Recovery Console, and checked all ssh configs, authorized_keys files, everything, and could not find an error.
What I did found was ssh brute force attack against the server, so I change the port.
Afterwards I start getting the message Connection Closed when trying to connect to the server via terminal.
Here I leave ssh_config, sshd_config, sources.list, the output of connecting with ssh, and the output of making apt update inside the recovery console.
ssh_config

# This is the ssh client system-wide configuration file.  See
# ssh_config(5) for more information.  This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.

# Configuration data is parsed as follows:
#  1. command line options
#  2. user-specific file
#  3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.

# Site-wide defaults for some commonly used options.  For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.

Host *
#bunch of commented options
    SendEnv LANG LC_*
    HashKnownHosts yes
    GSSAPIAuthentication yes

sshd_config

#   $OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

Port <PORT>
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

PubkeyAuthentication no

# Expect .ssh/authorized_keys2 to be disregarded by default in future.
#AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no 

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin yes
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM no

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

# override default of no subsystems
Subsystem   sftp    /usr/lib/openssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#   X11Forwarding no
#   AllowTcpForwarding no
#   PermitTTY no
#   ForceCommand cvs server

sources.list, now changed with groovy but originally changed with focal

## Note, this file is written by cloud-init on first boot of an instance
## modifications made here will not survive a re-bundle.
## if you wish to make changes you can:
## a.) add 'apt_preserve_sources_list: true' to /etc/cloud/cloud.cfg
##     or do the same in user-data
## b.) add sources in /etc/apt/sources.list.d
## c.) make changes to template file /etc/cloud/templates/sources.list.tmpl

# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://mirrors.digitalocean.com/ubuntu/ groovy main restricted
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy main restricted

## Major bug fix updates produced after the final release of the
## distribution.
deb http://mirrors.digitalocean.com/ubuntu/ groovy-updates main restricted
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy-updates main restricted

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://mirrors.digitalocean.com/ubuntu/ groovy universe
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy universe
deb http://mirrors.digitalocean.com/ubuntu/ groovy-updates universe
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy-updates universe

## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team, and may not be under a free licence. Please satisfy yourself as to
## your rights to use the software. Also, please note that software in
## multiverse WILL NOT receive any review or updates from the Ubuntu
## security team.
deb http://mirrors.digitalocean.com/ubuntu/ groovy multiverse
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy multiverse
deb http://mirrors.digitalocean.com/ubuntu/ groovy-updates multiverse
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy-updates multiverse

## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
deb http://mirrors.digitalocean.com/ubuntu/ groovy-backports main restricted universe multiverse
# deb-src http://mirrors.digitalocean.com/ubuntu/ groovy-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu groovy partner
# deb-src http://archive.canonical.com/ubuntu groovy partner

deb http://old-releases.ubuntu.com/ubuntu groovy-security main restricted
# deb-src http://old-releases.ubuntu.com/ubuntu groovy-security main restricted
deb http://old-releases.ubuntu.com/ubuntu groovy-security universe
# deb-src http://old-releases.ubuntu.com/ubuntu groovy-security universe
deb http://old-releases.ubuntu.com/ubuntu groovy-security multiverse
# deb-src http://old-releases.ubuntu.com/ubuntu groovy-security multiverse

on server, via recory console, sudo apt update

[...]
Err:5  groovy-updates Release
  404  Not Found [IP: 172.67.148.71 80]
Err:6  Release
  404  Not Found [IP: 172.67.148.71 80]
Hit:7  disco InRelease
Ign:8  disco InRelease
Err:9  disco Release
  404  Not Found [IP: 185.125.190.52 80]
[...]

on client, ssh -vvvv -i <key> <user>@<ip>

OpenSSH_8.2p1 Ubuntu-4ubuntu0.5, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /home/ivangk/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname <IP> is address
debug2: ssh_connect_direct
debug1: Connecting to <IP> [<IP>] port <PORT>.
debug1: Connection established.
[...]
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
Connection closed by <IP> port <PORT>

I am lost at what can be causing this problem. If I do lsof -i:<PORT> I get the correct port that I configured active, and ssh is active via systemctl status sshd.

vidarlo avatar
ar flag
19.04 has been end of life for three years. Upgrade. And deal with brute force attacks by disallowing password authentication and fail2ban, or similar - not obscurity.
ivangk avatar
ru flag
@vidarlo hi! I had password authentication disabled, but enabled it to see if the problem was the key pair authentication, alas, the problem remains
vidarlo avatar
ar flag
You probably have external firewall blocking your new port. But again: update to a sane version.
ivangk avatar
ru flag
hi @vidarlo thanks for the comments! nope, I have allowed the new port through ufw, and still this is happening. Should have added that to the post.
vidarlo avatar
ar flag
How about external firewalls?
ivangk avatar
ru flag
good question, I will check Digital Ocean for that, though there are other servers running without a problem... I think something of the focal update broke ssh
ivangk avatar
ru flag
@vidarlo sorry for the delay, happy new year! checked and the droplet has no firewall configured from DO... I think I'll recover the files and start with a clean one
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.