Why the RHEL8 system do not generate SSH host keys automatically when missing?

Jaroslav Kucera

11/24/23, 6:28 PM

On the RHEL 8 and previous it is usual, that the SSH host keys in /etc/ssh are generated automatically by sshd service when missing. Usually there should be:

/etc/ssh/ssh_host_ecdsa_key
/etc/ssh/ssh_host_ecdsa_key.pub
/etc/ssh/ssh_host_ed25519_key
/etc/ssh/ssh_host_ed25519_key.pub
/etc/ssh/ssh_host_rsa_key
/etc/ssh/ssh_host_rsa_key.pub

Restart of the node or even systemctl restart sshd should be sufficient.

But as of the minor version RHEL 8.7 this may not work any more and the sshd crashes complaining about missing host keys in the journal log. Why? How can I solve this?

2527

2 + 0

ssh

ssh-keys

cloud-init

rhel8

Score:5

Server

Jaroslav Kucera

11/24/23, 6:28 PM

The sshd service by default calls sshd-keygen.target, which checks availability of host keys in /etc/ssh directory and generates it when missing.

However this well known functionality can be blocked by the new version of cloud-init. As of cloud-init-22.1-5.el8.noarch there is new file:

/etc/systemd/system/[email protected]/disable-sshd-keygen-if-cloud-init-active.conf

with content:

# In some cloud-init enabled images the sshd-keygen template service may race
# with cloud-init during boot causing issues with host key generation.  This
# drop-in config adds a condition to [email protected] if it exists and
# prevents the sshd-keygen units from running *if* cloud-init is going to run.
#
[Unit]
ConditionPathExists=!/run/systemd/generator.early/multi-user.target.wants/cloud-init.target

So when you use the cloud-init you have 2 options now:

Generate host keys manually with ssh-keygen -A (see How to change a SSH host key? for more details and options.
Comment the condition

Simply put the # sign before ConditionPathExists...

[Unit]
#ConditionPathExists=!/run/systemd/generator.early/multi-user.target.wants/cloud-init.target

Then reload the systemd configuration with systemctl daemon-reload. The usual behavior should be working again.

+ 0

Score:1

Server

Arun S

1/18/24, 3:02 PM

If you are using cloud-init, then you can fix it by adding 'ssh' module in cloud-init config file in section "cloud_init_modules". Refer cloud-init docs

This will generate ssh host key during the firstboot.

You can test this in instance where you are having issue:

cloud-init config file: /etc/cloud/cloud.cfg
Check if you have 'ssh' module in "cloud_init_modules" section
Run this command to verify cloud-init action. NOTE: This will REBOOT your instance and run the cloud-init action from the scratch.
# cloud-init clean --reboot

Verify the ssh host key in /etc/ssh/ directory and sshd service status.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Why the RHEL8 system do not generate SSH host keys automatically when missing?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.