Score:1

Limit CPU and Memory using cgroup in Ubuntu 20.04 LTS server edition

in flag

I have 3 groups in Ubuntu 20.04 LTS server which is group1, group2, group3. I would like to limit CPU and memory for each groups.

  1. Group1 only allowed to use 20% CPU and 10GB RAM
  2. Group2 only allowed to use 30% CPU and 15GB RAM
  3. Group3 only allowed to use 50% CPU and 20GB RAM

Is this possible by using systemd and cgroup in 20.04 LTS server edition? Or is there any other method?

Score:1
cn flag

I know this may be late, but here's what worked for me. These steps were run on Pop! OS 20.04 (downstream from Ubuntu 20.04), but I think they'll also work on Ubuntu server. You might want to run all these commands in a super user (elevated) terminal. Just enter it using sudo su.

In the following, I'll set up cgroup limits such that users belonging to group cglims are allowed to only use 95% CPU and 28 GB of RAM. You can make changes accordingly (to add multiple groups, change destination names, limits, etc.). I use vim (vi), you can use any editor.

  1. Install cgroup-tools package

    apt install cgroup-tools -y
    
  2. Copy default configurations from the docs

    cp /usr/share/doc/cgroup-tools/examples/cgred.conf /etc/
    # Verify file
    view /etc/cgred.conf
    
  3. Create a file for cgroup constraints (configurations)

    vi /etc/cgconfig.conf
    

    Enter the following in the file

    group ucglims {
         cpu {
             cpu.cfs_quota_us=95000;
         }
         memory {
             memory.limit_in_bytes = 28g;
         }
    }
    

    The name ucglims is the destination (for mentioning constraints). It's not the group name.

  4. Create rules for mapping users, groups to resources and destinations.

    vi /etc/cgrules.conf
    

    Enter the following in the file

    #<user>        #<controllers>         #<destination>
    @cglims        cpu,memory             ucglims
    

    This means that the users in group cglims are going to be capped to the cpu and memory (RAM) limits in ucglims destination (described earlier).

  5. Parse and apply the configurations

    # Parse
    /usr/sbin/cgconfigparser -l /etc/cgconfig.conf
    # Apply
    /usr/sbin/cgrulesengd -vvv
    

    The above command activates the cgroup restrictions. You can test it. Say you have a test123 user in the group cglims. Login as that user and run

    # Login as test123 (shouldn't be elevated; open separate terminal)
    su test123
    # See if this is getting populated
    cat /sys/fs/cgroup/memory/ucglims/tasks
    cat /sys/fs/cgroup/cpu/ucglims/tasks
    # Run the following for 'test123' user. It should fail
    # WARN: This tries getting 40 GB RAM
    stress -m 4 --vm-bytes 10G --vm-keep -t 10
    

    The last command above should fail (coming from test123). At least, you'll see that after the 28GB limit on the RAM, it'll start using swap (instead of RAM). The process gets killed if it runs out of swap space (but leaving 28GB, the rest of the RAM remains free).

    You can use stress for other kinds of limit testing as well.

  6. We have to make this persistent (so that this is all not lost after reboots). We'll create systemd services for doing this.

    1. Create a service for parsing

      vi /etc/systemd/system/cgconfigparser.service
      

      Enter the following

      [Unit]
      Description=cgroup config parser
      After=network.target
      
      [Service]
      User=root
      Group=root
      ExecStart=/usr/sbin/cgconfigparser -l /etc/cgconfig.conf
      Type=oneshot
      
      [Install]
      WantedBy=multi-user.target
      
    2. Create a service for applying the rules

      vi /etc/systemd/system/cgrulesgend.service
      

      Enter the following

      [Unit]
      Description=cgroup rules generator
      After=network.target cgconfigparser.service
      
      [Service]
      User=root
      Group=root
      Type=forking
      EnvironmentFile=-/etc/cgred.conf
      ExecStart=/usr/sbin/cgrulesengd
      Restart=on-failure
      
      [Install]
      WantedBy=multi-user.target
      
    3. Now reload the systemd daemon, enable the services, and start them

      # Reload context daemon to 'discover' everything
      sudo systemctl daemon-reload
      # Enable services to run on boot
      sudo systemctl enable cgconfigparser
      sudo systemctl enable cgrulesgend
      # Start them
      sudo systemctl start cgconfigparser
      sudo systemctl start cgrulesgend
      

Done!

You may add more entries in steps 3 and 4 to suit your needs, but I think the above should work.

I have duct taped this solution from the following sources

avneesh mishra avatar
cn flag
You can use tools like `systemd-cgtop` to monitor different destination (group) usage.
avneesh mishra avatar
cn flag
Also, note that the CPU percentage is relative to the number of threads. Say, for example, you have an 8-thread CPU; setting cgroup to 700% will mean that you can use up to 7 threads at full (the actual usage will be distributed, but the cap is 800%). In my example, it's 95% (less than a single thread).
Routhinator avatar
in flag
Anyone have a cgroups v2 update for this for 22.04+ ?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.