Join Log in
Score:0
eijeze avatar Server

SSHd on Fedora 36 doesn't start after boot, even after systemctl enable sshd.service

in flag
eijeze

When I boot my desktop sshd is not running:

=$ systemctl status sshd.service
○ sshd.service - OpenSSH server daemon
     Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: disabled)
     Active: inactive (dead)
       Docs: man:sshd(8)
             man:sshd_config(5)

If I'll do sudo systemctl enable sshd.service; sudo systemctl start sshd.service it comes online without problem and status shows:

=$ sudo systemctl status sshd.service
● sshd.service - OpenSSH server daemon
     Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: disabled)
     Active: active (running) since Tue 2022-10-25 12:25:47 CEST; 1min 11s ago
       Docs: man:sshd(8)
             man:sshd_config(5)
   Main PID: 14091 (sshd)
      Tasks: 1 (limit: 76311)
     Memory: 2.3M
        CPU: 8ms
     CGroup: /system.slice/sshd.service
             └─ 14091 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups"
 
Oct 25 12:25:47 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 25 12:25:47 orange.home.depesz.com sshd[14091]: Server listening on 0.0.0.0 port 22.
Oct 25 12:25:47 orange.home.depesz.com sshd[14091]: Server listening on :: port 22.
Oct 25 12:25:47 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.

But on next reboot - it's still down.

Last lines from journalctl | grep sshd.service don't show anything bad:

Oct 18 11:45:20 orange.home.depesz.com sudo[4936]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl enable sshd.service
Oct 18 11:45:27 orange.home.depesz.com sudo[5039]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl start sshd.service
Oct 18 11:45:28 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 18 11:45:28 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 18 11:51:40 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 18 11:51:41 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 18 11:51:41 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 18 11:56:37 orange.home.depesz.com sudo[3079]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl enable sshd.service
Oct 18 11:56:40 orange.home.depesz.com sudo[3114]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl start sshd.service
Oct 18 11:56:40 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 18 11:56:40 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 18 11:56:49 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 18 11:56:49 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 18 11:56:50 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 18 11:57:18 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 18 11:57:18 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 18 12:26:50 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 18 12:26:50 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 18 12:26:50 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 18 12:31:42 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 18 12:31:42 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 18 12:51:31 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 18 12:51:31 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 18 12:51:31 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 18 13:30:04 orange.home.depesz.com baloo_file_extractor[3719]: kf.baloo: "/home/depesz/tmp/old-orange/var/lib/systemd/deb-systemd-helper-enabled/sshd.service" id seems to have changed. Perhaps baloo was not running, and this file was deleted + re-created
Oct 18 17:02:51 orange.home.depesz.com sudo[41862]:   depesz : TTY=pts/5 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl start sshd.service
Oct 18 17:02:51 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 18 17:02:51 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 19 14:02:24 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 19 14:02:24 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 19 14:02:24 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 19 14:02:24 orange.home.depesz.com systemd[1]: sshd.service: Consumed 1.732s CPU time.
Oct 19 14:05:10 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 19 14:05:10 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 23 10:25:58 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 23 10:25:58 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 24 20:57:45 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 24 20:57:45 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 24 20:57:45 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 24 20:57:45 orange.home.depesz.com systemd[1]: sshd.service: Consumed 2.378s CPU time.
Oct 25 07:50:08 orange.home.depesz.com sudo[8157]:   depesz : TTY=pts/10 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl enable sshd.service
Oct 25 07:50:10 orange.home.depesz.com sudo[8275]:   depesz : TTY=pts/10 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl start sshd.service
Oct 25 07:50:10 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 25 07:50:10 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 25 11:44:24 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 25 11:44:24 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 25 11:44:24 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 25 12:25:37 orange.home.depesz.com sudo[13952]:   depesz : TTY=pts/2 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl enable sshd.service
Oct 25 12:25:47 orange.home.depesz.com sudo[14088]:   depesz : TTY=pts/2 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl start sshd.service
Oct 25 12:25:47 orange.home.depesz.com systemd[1]: Starting sshd.service - OpenSSH server daemon...
Oct 25 12:25:47 orange.home.depesz.com systemd[1]: Started sshd.service - OpenSSH server daemon.
Oct 25 12:26:30 orange.home.depesz.com sudo[14226]:   depesz : TTY=pts/2 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/journalctl _SYSTEMD_UNIT=sshd.service
Oct 25 12:26:58 orange.home.depesz.com sudo[14388]:   depesz : TTY=pts/2 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl status sshd.service
Oct 25 12:27:48 orange.home.depesz.com systemd[1]: Stopping sshd.service - OpenSSH server daemon...
Oct 25 12:27:48 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.
Oct 25 12:27:48 orange.home.depesz.com systemd[1]: Stopped sshd.service - OpenSSH server daemon.
Oct 25 12:40:37 orange.home.depesz.com sudo[7448]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl status sshd.service
Oct 25 12:40:48 orange.home.depesz.com sudo[7604]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl status sshd.service
Oct 25 12:40:54 orange.home.depesz.com sudo[7672]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl status sshd.service
Oct 25 12:41:15 orange.home.depesz.com sudo[7772]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/systemctl status sshd.service
Oct 25 12:41:49 orange.home.depesz.com sudo[7988]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/journalctl _SYSTEMD_UNIT=sshd.service
Oct 25 12:42:20 orange.home.depesz.com sudo[8147]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/journalctl _SYSTEMD_UNIT=sshd.service -n 100
Oct 25 12:42:25 orange.home.depesz.com sudo[8214]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/journalctl _SYSTEMD_UNIT=sshd.service -n 50
Oct 25 12:42:28 orange.home.depesz.com sudo[8286]:   depesz : TTY=pts/1 ; PWD=/home/depesz ; USER=root ; COMMAND=/usr/bin/journalctl _SYSTEMD_UNIT=sshd.service -n 50

Any idea on how to fix/debug it?

Ah, as final note - there is nothing listening on :22 after reboot. And I can start sshd.service without any problem using systemctl start sshd.service

463
1 + 1
ssh
Göran Uddeborg avatar
in flag
Göran Uddeborg
There might be clues on what is going on in the lines you remove with your `grep`. Show all journal entries from a start to a stop, with some margin, e.g. `journalctl --since 11:56:35 --until 11:56:55`.
0
Reply
Score:1
Tilman Schmidt avatar Server
bd flag
Tilman Schmidt

Your log shows the sshd.service being actively deactivated, for example here:

Oct 25 12:27:48 orange.home.depesz.com systemd[1]: sshd.service: Deactivated successfully.

So someone or something is doing sudo systemctl disable sshd.service, thereby instructing systemd not to start sshd during the next system startup.

You need to investigate who or what is doing that, and fix it accordingly.

+ 7
eijeze avatar
in flag
eijeze
Any idea on how? It is my desktop machine, noone uses it but me, and I definitely didn't disable sshd
0
Reply
Tilman Schmidt avatar
bd flag
Tilman Schmidt
The usual way in a business environment: activate and check logs, correlate times with other system activities, check those other activities. But your comment seems to imply that this is not a business environment, and hence offtopic on Server Fault.
0
Reply
eijeze avatar
in flag
eijeze
Not entirely sure what business environemnt has to do with anything - it is a server, so serverfault should be relevant? Or am I missing something? Anyway - I'll try to figure it out, too bad that there isn't any trail that would tell me what caused that :(
0
Reply
Tilman Schmidt avatar
bd flag
Tilman Schmidt
The official topic definition of Server Fault is: "questions about managing information technology systems in a business environment"
0
Reply
eijeze avatar
in flag
eijeze
Ah, sorry then. What would be the place to ask about linux/unix-related issues at home? Also, where can i see this "official topic definition"? I don't see text anywhere on https://serverfault.com/
0
Reply
Tilman Schmidt avatar
bd flag
Tilman Schmidt
The topic definition is at https://serverfault.com/help/on-topic . An appropriate site for home IT questions would be https://superuser.com .
0
Reply
eijeze avatar
in flag
eijeze
OK. thanks. will try to remember.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.