Score:0

Avoid systemd user scope kill on shutdown

lt flag

I create a Debian Bookworm machine to serve as LXC containers manager. I use unprivileged LXC containers started with lxc-unpriv-start command that create a systemd user scope. I create a service that start my containers on server start and launch a clean shutdown on service stop. Every container can take 2 or 3 minutes to shutdown. If I stop main service manually all works well. The problem arise when I shutdown the server because my service correctly wait for containers termination but meanwhile systemd-logind kill all user session brutally killing my container also.

I simulate the situation creating a sleep only shell and execute it with

/usr/bin/systemd-run --user --scope -p "Delegate=yes" wait.sh &

and it get killed on shutdown. How can I avoid this kill?

Tom Yan avatar
in flag
Can you share the service file? And is the service itself a system service or a user service (i.e. which you manipulate with `systemctl --user`)?
RoseIV avatar
lt flag
Hi Tom. There is no service file. lxc-unpriv-start execute a systemd-run command (like the one I showed above) that create a systemd scope, not a service.
Tom Yan avatar
in flag
So you always run the command directly from a shell? In that case it means systemd will have no idea about what stop command should be run before it stops the user manager.
Score:0
in flag

Assuming the container-stopping command "blocks" (i.e., exits after the container has been stopped), you can probably run the container-starting command with a systemd user service instead of directly from a shell. Suppose you need to start multiple containers, you can write a service template to $HOME/.config/systemd/user/. The filename of a service template should have @ before .service (e.g. [email protected]).

Here's an example for you:

[Unit]
PartOf=%i.scope
After=%i.scope

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/systemd-run --user --scope -u %i sh -c 'sleep 1d &'
ExecStop=/usr/bin/sleep 1m

As you can see, the scope name needs to be "known" instead of a generated one.

With the template, you can now start the container with e.g. systemctl --user start test@container-a. You should now see the corresponding scope remain until its "agent" service has stopped (i.e., the ExecStop= command has exited).

You might need to run systemctl --user daemon-reload whenever you have made changes in the service (template) file(s).

P.S. Obviously if you need to pass extra container-specific arguments (that are not the same string as the scope name) to the container-starting/stopping command, you probably have to write multiple service files instead of using a template.

RoseIV avatar
lt flag
This is exactly what I'm testing right now. I have to personalize "[email protected]" to increase TimeoutSec but it works. The only problem remaining is that I can't add a Require directive for lxc system service. In this way when lxc stop gives error because in use.
Score:0
mx flag

Linger should solve that problem.

From https://www.freedesktop.org/software/systemd/man/loginctl.html

Enable/disable user lingering for one or more users. If enabled for a specific user, a user manager is spawned for the user at boot and kept around after logouts. This allows users who are not logged in to run long-running services. Takes one or more user names or numeric UIDs as argument. If no argument is specified, enables/disables lingering for the user of the session of the caller.

loginctl enable-linger <username>

Enabling linger for the user session will allow your containers to continue running even after you log out or during system shutdown. This should prevent systemd-logind from forcefully terminating the user session and the associated containers, giving them the opportunity to shut down gracefully.

RoseIV avatar
lt flag
Hi Grant. I have already activated linger for user (lxc want it) but shutdown seems to ignore it. I also added KillUserProcesses=no on logind.conf but nothing is changed.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.