we have two Ubuntu 18-04 servers and they both lose their Failover IP settings around 06 and 07AM.
They have always worked perfectly, in fact, both installations are new. One done 1 month ago and one done about 2 months ago.
In journalctl, I noticed this interesting thing:
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 sshd[13559]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=47.243.66.79 user=root
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 dbus-daemon[1264]: [system] Reloaded configuration
Jul 22 07:30:39 ns1000 sshd[13462]: Failed password for root from 8.210.200.149 port 55222 ssh2
Jul 22 07:30:39 ns1000 systemd[1]: Reloading.
Jul 22 07:30:40 ns1000 sshd[13462]: Connection closed by authenticating user root 8.210.200.149 port 55222 [preauth]
Jul 22 07:30:40 ns1000 systemd[1]: Reexecuting.
Jul 22 07:30:40 ns1000 systemd[1]: systemd 237 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +E
Jul 22 07:30:40 ns1000 systemd[1]: Detected architecture x86-64.
Jul 22 07:30:40 ns1000 systemd[1]: Stopped Wait for Network to be Configured.
Jul 22 07:30:40 ns1000 systemd[1]: Stopping Wait for Network to be Configured...
Jul 22 07:30:40 ns1000 systemd[1]: Stopping Network Service...
Jul 22 07:30:40 ns1000 systemd-timesyncd[6618]: Network configuration changed, trying to establish connection.
Jul 22 07:30:40 ns1000 systemd[1]: Stopped Network Service.
Jul 22 07:30:40 ns1000 systemd[1]: Starting Network Service...
Jul 22 07:30:40 ns1000 systemd-networkd[14213]: enp1s0f0: Gained IPv6LL
Jul 22 07:30:40 ns1000 systemd-networkd[14213]: Enumeration completed
Jul 22 07:30:40 ns1000 systemd-timesyncd[6618]: Network configuration changed, trying to establish connection.
Jul 22 07:30:40 ns1000 systemd[1]: Started Network Service.
Jul 22 07:30:40 ns1000 systemd-networkd[14213]: enp1s0f0: IPv6 successfully enabled
Jul 22 07:30:40 ns1000 systemd[1]: Starting Wait for Network to be Configured...
Jul 22 07:30:40 ns1000 systemd[1]: Stopping Network Name Resolution...
Jul 22 07:30:40 ns1000 systemd[1]: Stopped Network Name Resolution.
Jul 22 07:30:40 ns1000 systemd-networkd[14213]: enp1s0f0: Configured
Jul 22 07:30:40 ns1000 systemd-networkd-wait-online[14225]: managing: enp1s0f0
Jul 22 07:30:40 ns1000 systemd-networkd-wait-online[14225]: ignoring: lo
Jul 22 07:30:40 ns1000 systemd[1]: Starting Network Name Resolution...
Jul 22 07:30:40 ns1000 systemd[1]: Started Wait for Network to be Configured.
Jul 22 07:30:40 ns1000 systemd-resolved[14239]: Positive Trust Anchors:
Jul 22 07:30:40 ns1000 systemd-resolved[14239]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
Jul 22 07:30:40 ns1000 systemd-resolved[14239]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Jul 22 07:30:40 ns1000 systemd-resolved[14239]: Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172
Jul 22 07:30:40 ns1000 systemd-resolved[14239]: Using system hostname 'ns1000'.
Jul 22 07:30:40 ns1000 systemd[1]: Started Network Name Resolution.
Jul 22 07:30:40 ns1000 systemd[1]: Stopping Network Time Synchronization...
Jul 22 07:30:41 ns1000 systemd[1]: Stopped Network Time Synchronization.
Jul 22 07:30:41 ns1000 systemd[1]: Starting Network Time Synchronization...
Jul 22 07:30:41 ns1000 systemd[1]: Started Network Time Synchronization.
This happened right after starting the upgrade of some packages suggested by Webmin:
Start-Date: 2021-07-22 07:30:38
Commandline: apt-get -y install apt apt-transport-https apt-utils gzip initramfs-tools initramfs-tools-bin initramfs-tools-core libapt-inst2.0 libapt-pkg5.0 libnss-systemd libpam-systemd libsystemd0 libudev1 linux-base systemd systemd-sysv udev
Upgrade: libapt-inst2.0:amd64 (1.6.13, 1.6.14), libsystemd0:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), apt:amd64 (1.6.13, 1.6.14), linux-base:amd64 (4.5ubuntu1.2, 4.5ubuntu1.6), udev:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), libapt-pkg5.0:amd64 (1.6.13, 1.6.14), initramfs-tools-bin:amd64 (0.130ubuntu3.12, 0.130ubuntu3.13), libudev1:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), systemd-sysv:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), libpam-systemd:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), systemd:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), apt-utils:amd64 (1.6.13, 1.6.14), libnss-systemd:amd64 (237-3ubuntu10.49, 237-3ubuntu10.50), apt-transport-https:amd64 (1.6.13, 1.6.14), gzip:amd64 (1.6-5ubuntu1, 1.6-5ubuntu1.1), initramfs-tools-core:amd64 (0.130ubuntu3.12, 0.130ubuntu3.13), initramfs-tools:amd64 (0.130ubuntu3.12, 0.130ubuntu3.13)
End-Date: 2021-07-22 07:30:54
However, we did the package update because yesterday both dedicates had the same problem, they are the only Ubuntu 18-04 ones we have. Actually, one of the dedicates gave the problem again today around 06:30AM, so we did the package update on both of them. Well, I checked that this can be caused by a bug in systemd which is fixed in version 238, however Ubuntu 18-04 doesn't have it. Both use version 237.
The addition of the IPs was done in one of the dedicated ones by ip addr add ip/32 dev bridge and in the other one directly by Webmin.
Does anyone have any suggestions?
