Score:0

How to dynamically change cgroup of systemd service subprocesses (gitlab runner)?

ru flag

I am running a gitlab runner and I want each job to put itself into its own cgroup.

I can set the gitlab bash subprocess to the right cgroup, but its children don't inherit the cgroup.

This is how I'm doing it:

$ /usr/bin/ps -opid,cgroup $PPID $$
   PID CGROUP
 43547 11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/user.slice,1:name=systemd:/user.slice/user-988.slice/session-c2500.scope
 43548 11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/user.slice,1:name=systemd:/user.slice/user-988.slice/session-c2500.scope
$ /usr/bin/sudo /usr/bin/env CGROUP_LOGLEVEL=DEBUG /usr/bin/cgclassify -g cpu,cpuacct:/gitlab-runner/$CI_CONCURRENT_PROJECT_ID --sticky $PPID $$
found cpu in rw,nosuid,nodev,noexec,relatime,cpuacct,cpu
Found cgroup option rw,nosuid,nodev,noexec,relatime,cpuacct,cpu, count 0
...trimmed...
Will move pid 43547 to cgroup '/gitlab-runner/0'
Adding controller cpu
Adding controller cpuacct
Will move pid 43548 to cgroup '/gitlab-runner/0'
Adding controller cpu
Adding controller cpuacct
$ /usr/bin/ps -opid,cgroup $PPID $$
   PID CGROUP
 43547 11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/gitlab-runner/0,1:name=systemd:/user.slice/user-988.slice/session-c2500.scope
 43548 11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/gitlab-runner/0,1:name=systemd:/user.slice/user-988.slice/session-c2500.scope

Querying a different run, finding the processes in the cgroup:

$ ps -e -opid,comm,cgroup | grep gitlab-runner/3
 77554 su              11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/gitlab-runner/3,1:name=systemd:/user.slice/user-988.slice/session-c2604.scope
 77555 bash            11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/gitlab-runner/3,1:name=systemd:/user.slice/user-988.slice/session-c2604.scope

Looking at its children...

$ pstree -p 77554
su(77554)───bash(77555)───bash(77575)───python3.6(78199)───run-cypress-spl(78206)───coverage(80245)─┬─node(91561)─┬─Cypress(91796)─┬─Cypress(91799)───Cypress(91990)─┬─{Cypress}(91992)
                                                                                                    │             │                │                                 ├─{Cypress}(91993)
  

Its children aren't in the cgroup:

$ ps  -opid,comm,cgroup 78206 | cat
   PID COMMAND         CGROUP
   78206 run-cypress-spl  78206 run-cypress-spl 11:blkio:/user.slice,9:devices:/user.slice,7:pids:/user.slice,6:memory:/user.slice,2:cpuacct,cpu:/user.slice,1:name=systemd:/user.slice/user-988.slice/session-c2604.scope

I also found https://stackoverflow.com/questions/50749408/how-systemd-tracks-fork-process-with-type-fork which implies that systemd might prevent cgroup inheritance at fork.

Is there a way around this?

CentOS7, systemd-219-78.el7_9.5.x86_64.

Update: If I understand this answer, the answer may be Delegate=true. I don't think this is supported on this systemd... I'm trying it out next.

Update: I tried this, and no difference was found:

$ cat /etc/systemd/system/gitlab-runner.service.d/override.conf
[Service]
Delegate=yes

Update: this may be an alternate way I will try next: https://unix.stackexchange.com/questions/490978/limit-cpu-and-memory-consumption-of-all-x-applications

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.