Join Log in
Score:3
Evy Bongers avatar Server

How can I run a systemd service after network link has been established?

us flag
Evy Bongers

A while ago I set up a systemd service to run after network comes online per the documentation. This is the unit file:

[Unit]
Description=Refresh Pacman mirrorlist with Reflector.
Documentation=https://wiki.archlinux.org/index.php/Reflector
Wants=network-online.target
After=network-online.target nss-lookup.target

[Service]
Type=oneshot
ExecStart=/usr/bin/reflector @/etc/xdg/reflector/reflector.conf
CacheDirectory=reflector
CapabilityBoundingSet=~CAP_SETUID CAP_SETGID CAP_SETPCAP CAP_SYS_ADMIN CAP_SYS_PTRACE CAP_CHOWN CAP_FSETID CAP_SETFCAP CAP_DAC_OVERRIDE CAP_DAC_READ_SEARCH CAP_FOWNER CAP_IPC_OWNER CAP_NET_ADMIN CAP_SYS_TIME CAP_AUDIT_CONTROL CAP_AUDIT_READ CAP_AUDIT_WRITE CAP_KILL CAP_NET_BIND_SERVICE CAP_NET_BROADCAST CAP_NET_RAW CAP_SYS_NICE CAP_SYS_RESOURCE CAP_MAC_ADMIN CAP_MAC_OVERRIDE CAP_SYS_BOOT CAP_LINUX_IMMUTABLE CAP_IPC_LOCK CAP_SYS_CHROOT CAP_BLOCK_SUSPEND CAP_LEASE CAP_SYS_PACCT CAP_SYS_TTY_CONFIG CAP_WAKE_ALARM
Environment=XDG_CACHE_HOME=/var/cache/reflector
LockPersonality=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
PrivateUsers=true
ProtectClock=true
ProtectControlGroups=true
ProtectHome=true
ProtectHostname=true
ProtectKernelTunables=true
ProtectKernelLogs=true
ProtectKernelModules=true
ProtectSystem=strict
ReadOnlyPaths=/etc/xdg/reflector/reflector.conf
ReadWritePaths=/etc/pacman.d/mirrorlist
RemoveIPC=true
RestrictAddressFamilies=~AF_AX25 AF_IPX AF_APPLETALK AF_X25 AF_DECnet AF_KEY AF_NETLINK AF_PACKET AF_RDS AF_PPPOX AF_LLC AF_IB AF_MPLS AF_CAN AF_TIPC AF_BLUETOOTH AF_ALG AF_VSOCK AF_KCM AF_UNIX AF_XDP
RestrictNamespaces=true
RestrictRealtime=true
RestrictSUIDSGID=true
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@resources @privileged
UMask=177

[Install]
WantedBy=multi-user.target

Per the documentation, the service should already be set up correctly to wait for network connectivity to be established (i.e.: I'm logged in, NetworkManager has connected to WiFi and the system has been assigned an IP).

network-online.target is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind. Its primary purpose is to actively delay activation of services until the network is set up.

I've also made sure that NetworkManager-wait-online.service is enabled.

$ systemctl is-enabled NetworkManager-wait-online.service systemd-networkd-wait-online.service
enabled
disabled

Instead of the service waiting to start until an internet connection has been established, I see an error on the system console almost as soon as I see my login prompt.

Jul 05 07:40:02 gadget systemd[1]: Starting Refresh Pacman mirrorlist with Reflector....
Jul 05 07:40:03 gadget reflector[564]: error: failed to retrieve mirrorstatus data: URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
Jul 05 07:40:03 gadget systemd[1]: reflector.service: Main process exited, code=exited, status=1/FAILURE
Jul 05 07:40:03 gadget systemd[1]: reflector.service: Failed with result 'exit-code'.
Jul 05 07:40:03 gadget systemd[1]: Failed to start Refresh Pacman mirrorlist with Reflector..

Since it should wait for network to be online and DNS to be available, I also checked the status of NetworkManager-wait-online.service, network-online.target and nss-lookup.target:

$ systemctl status NetworkManager-wait-online.service
● NetworkManager-wait-online.service - Network Manager Wait Online
     Loaded: loaded (/usr/lib/systemd/system/NetworkManager-wait-online.service; enabled; vendor preset: disabled)
     Active: active (exited) since Mon 2021-07-05 07:40:02 CEST; 44min ago
       Docs: man:nm-online(1)
   Main PID: 544 (code=exited, status=0/SUCCESS)
      Tasks: 0 (limit: 38321)
     Memory: 0B
        CPU: 0
     CGroup: /system.slice/NetworkManager-wait-online.service

Jul 05 07:40:02 gadget systemd[1]: Starting Network Manager Wait Online...
Jul 05 07:40:02 gadget systemd[1]: Finished Network Manager Wait Online.
$ systemctl status network-online.target
● network-online.target - Network is Online
     Loaded: loaded (/usr/lib/systemd/system/network-online.target; static)
     Active: active since Mon 2021-07-05 07:40:02 CEST; 1h 5min ago
       Docs: man:systemd.special(7)
             https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget

Jul 05 07:40:02 gadget systemd[1]: Reached target Network is Online.
$ systemctl status nss-lookup.target
○ nss-lookup.target - Host and Network Name Lookups
     Loaded: loaded (/usr/lib/systemd/system/nss-lookup.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)

It strikes me as odd that NetworkManager-wait-online.service reports finished right after starting. Is there something else I need to configure? Is this a bug in NetworkManager and/or nm-online?

I've looked for similar topics and found these questions, but they don't answer my issue:

3625
2 + 1
Michael Hampton avatar
cz flag
Michael Hampton
It already comes with its own systemd units though! Why not just use that?
0
Reply
Score:1
user10489 avatar Server
nc flag
user10489

The meaning of "network online" is nebulous and network-online.target only fufills the most basic of possible meanings of "network online". See https://freedesktop.org/wiki/Software/systemd/NetworkTarget for a more complete explanation.

You need to determine precisely what you need to be up in the network and what services need to be working for your service to function correctly and either add those as dependencies, or add tests in your service startup sequence to wait for them.

As the page above states at the end, best would be to adjust your service to be able to handle dynamic changes to network configuration rather than failing if it isn't totally as expected.

0 + 2
Evy Bongers avatar
us flag
Evy Bongers
I already linked those docs and quoted from them: "`network-online.target` is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind." So, if I understand correctly, the problem is that my expectation of the target does not align with the way the NetworkManager developers implemented it.
0
Reply
user10489 avatar
nc flag
user10489
network-online.target may be necessary but not sufficient. Maybe you need to also test that a specific hostname is resolvable or something Also, does this have to be oneshot, or can it be simple instead so that it doesn't block the rest of system startup?
0
Reply
Score:-1
Gediz GÜRSU avatar Server
in flag
Gediz GÜRSU

Check systemd-resolvd.service for Before=nss-lookup.target dependency. It should work.

Then check again to be sure :

systemctl is-enabled NetworkManager-wait-online.service systemd-networkd-wait-online.service
0 + 4
Evy Bongers avatar
us flag
Evy Bongers
`systemd-resolvd.service` has these listed: `Before=network.target nss-lookup.target shutdown.target`. Updated the original question to include the command.
0
Reply
Evy Bongers avatar
us flag
Evy Bongers
On second thought, why would that be related? I don't use `systemd-resolved.service`.
0
Reply
user10489 avatar
nc flag
user10489
If you don't use systemd-resolved.service, what resolver do you use? Perhaps you should add that to your dependencies.
0
Reply
Evy Bongers avatar
us flag
Evy Bongers
I use NetworkManager for network management. That's why I mentioned `NetworkManager-wait-online.service` several times, in contrast to `systemd-networkd-wait-online.service`.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.