Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to achieve resource restriction through Cgroups mechanism". In daily operation, I believe that many people have doubts about how to achieve resource restriction through Cgroups mechanism. The editor has consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to achieve resource restriction through Cgroups mechanism". Next, please follow the editor to study!

Cgroups functions and core concepts

Cgroups (full name: control groups) is a function of the Linux kernel, which can restrict the resources of a process or process group (such as CPU, memory, disk IO, etc.).

In 2006, Google engineers (Rohit Seth and Paul Menage were the main sponsors) launched the project, which was originally called a process container (process containers) instead of cgroups. In 2007, cgroups code was planned to be incorporated into the Linux kernel, but at that time in the Linux kernel, the word container was widely used and had different meanings. To avoid naming confusion and ambiguity, the process container was renamed cgroups and successfully incorporated into Linux version 2.6.24 in 2008. At present, cgroups has become the basis of systemd, Docker, Linux Containers (LXC) and other technologies.

Cgroups mainly provides the following functions:

Resource restrictions: limit the amount of resources used. For example, we can protect the safe operation of other businesses of the host by limiting the memory limit of a business.

Priority control: different groups can have different priorities for using resources (CPU, disk IO, etc.).

Audit: calculate the resource usage of the control group.

Control: controls the suspension or resumption of a process.

The realization of cgroups function depends on three core concepts: subsystem, control group and hierarchical tree.

Subsystem: a component of a kernel, and a subsystem represents a class of resource scheduling controllers. For example, the memory subsystem can limit the amount of memory used, and the CPU subsystem can limit the usage time of CPU.

Control group (cgroup): represents the relationship between a set of processes and a set of subsystems with parameters. For example, if a process uses the CPU subsystem to limit the time it takes to use CPU, the association between the process and the CPU subsystem is called a control group.

Hierarchical tree (hierarchy): consists of a series of control groups arranged according to the tree structure. This arrangement allows the control group to have a parent-child relationship, and the child control group has the properties of the parent control group by default, that is, the child control group inherits from the parent control group. For example, a control group C1 is defined in the system, which limits the use of 1 core for CPU, and then another control group c2 wants to limit the use of 1 core for CPU and 2G of memory at the same time, then c2 can directly inherit C1 without having to redefine the CPU limit.

Of the three core concepts of cgroups, the subsystem is the most core concept, because the subsystem is the basis for truly realizing the limitations of certain types of resources.

Cgroups subsystem instance

Let me demonstrate which subsystems are started by default on Linux through an example. Let's first check the cgroups information that has been mounted on the current system through the mount command:

Root@cr7-ubuntu:~# mount-t cgroupcgroup on / sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd) cgroup on / sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on / sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on / sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime Perf_event) cgroup on / sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on / sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on / sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on / sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on / sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime Pids) cgroup on / sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on / sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on / sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)

From the output, we can see that the current system has mounted our commonly used cgroups subsystems, such as cpu, memory, pids and other cgroups subsystems. Among these subsystems, the cpu and memory subsystems are the most frequently used subsystems in the container environment, which I will introduce in detail below.

Cpu subsystem

First, we execute a script like this in the background without limiting CPU usage:

Root@cr7-ubuntu:~# while:; do:; done & [1] 839960

Obviously, it executes an endless loop that can eat up to 100% of the computer's CPU, and according to its output, we can see that the process number (PID) of this script running in the background is 839960.

We can use the top command to confirm that the CPU is full:

As you can see in the output, CPU usage is already 100% (% Cpu (s): 98.0 us).

Now let's take the cpu subsystem as an example to demonstrate how cgroups limits the cpu usage time of a process. Since many cgroups operations require root permissions, we need to make sure that we have switched to the root user before executing the command. The following commands are executed using the root user by default.

Step 1: creating a cgroup cgroups under the cpu subsystem is easy. You only need to create a directory under the corresponding subsystem. Let's go to the cpu subsystem to create a test folder:

Root@cr7-ubuntu:~# mkdir / sys/fs/cgroup/cpu/mydocker

This directory is called a "control group". You will find that the operating system will automatically generate the resource limit files corresponding to the subsystem in your newly created container directory:

Root@cr7-ubuntu:~# ls-l / sys/fs/cgroup/cpu/mydockertotal 0 cpuacct.stat-rw-r--r-- RWKE RFT-1 root root 0 Mar 10 09:34 cgroup.clone_children-rw-r--r-- 1 root root 0 Mar 10 09:34 cgroup.procs-r--r--r-- 1 root root 0 Mar 10 09:34 cpuacct.stat-rw-r--r-- 1 root root 0 Mar 10 09:34 cpuacct.usage-r--r--r-- 1 Root root 0 Mar 10 09:34 cpuacct.usage_all-r--r--r-- 1 root root 0 Mar 10 09:34 cpuacct.usage_percpu-r--r--r-- 1 root root 0 Mar 10 09:34 cpuacct.usage_percpu_sys-r--r--r-- 1 root root 0 Mar 10 09:34 cpuacct.usage_percpu_user-r--r--r-- 1 root root 0 Mar 10 09:34 cpuacct.usage_sys-r--r-- Root root-1 root root 0 Mar 10 09:34 cpuacct.usage_user-rw-r--r-- 1 root root 0 Mar 10 09:34 cpu.cfs_period_us-rw-r--r-- 1 root root 0 Mar 10 09:34 cpu.cfs_quota_us-rw-r--r-- 1 root root 0 Mar 10 09:34 cpu.shares-r--r--r-- 1 root root 0 Mar 10 09:34 cpu.stat-rw-r--r-- 1 Root root 0 Mar 10 09:34 cpu.uclamp.max-rw-r--r-- 1 root root 0 Mar 10 09:34 cpu.uclamp.min-rw-r--r-- 1 root root 0 Mar 10 09:34 notify_on_release-rw-r--r-- 1 root root 0 Mar 10 09:34 tasks

You will notice keywords like cfs_period and cfs_quota in its output. These two parameters need to be combined and can be used to limit the amount of CPU time that a process can only be assigned to a total cfs_quota for a period of time with a length of cfs_period. If you subtract the scheduling cycle (that is, cpu.cfs_period_us) with this value, 50ms/100ms = 0. 5, then the maximum CPU quota allowed by this control group is 0. 5 CPU. You can see here that cpu.cfs_quota_us is an absolute value. If this value is 200000, which is 200ms, then it is divided by period, which is 200ms/100ms=2, and the result is more than 1 CPU, which means that the control group needs a resource quota of 2 CPU.

At this point, by looking at the files in the mydocker directory, we can see that there is no limit on CPU quota in the mydocker control group (that is,-1), while CPU period defaults to 100ms (100000 us):

Root@cr7-ubuntu:~# cat / sys/fs/cgroup/cpu/mydocker/cpu.cfs_quota_us-1root@cr7-ubuntu:~# cat / sys/fs/cgroup/cpu/mydocker/cpu.cfs_period_us100000

Step 2: set limit parameters

Next, we can set the limit by modifying the contents of these files. Write 20 ms (20000 us) to the cfs_quota file in the mydocker group:

Root@cr7-ubuntu:~# echo 20000 > / sys/fs/cgroup/cpu/mydocker/cpu.cfs_quota_us

It means that processes restricted by this control group can only use 20 ms of CPU time per 100 ms, which means that the process can only use 20% of the CPU bandwidth. (if CPU is multicore, then 20% is the sum of the bandwidth that the process can use in all CPU, for example, CPU1 is 10% and CPU 2 is 10%)

Step 3: join the process to the cgroup control group

Next, we write the PID of the restricted process to the tasks file in the mydocker group, and the above settings will take effect on the process:

Root@cr7-ubuntu:~# echo 839960 > / sys/fs/cgroup/cpu/mydocker/tasks

At this point, let's take a look with the top command: you can see that the computer's CPU usage immediately dropped to 20% (% Cpu0: 21.5us).

Memroy subsystem

Step 1: create a cgroup under the memory subsystem

Root@cr7-ubuntu:~# mkdir / sys/fs/cgroup/memory/mydocker

Again, let's take a look at the automatically created files in the newly created directory:

Root@cr7-ubuntu:~# ls-l / sys/fs/cgroup/memory/mydockertotal 0 cgroup.procs-rw-r--r-- 0 Mar 10 09:50 cgroup.clone_children--w--w--w- 1 root root 0 Mar 10 09:50 cgroup.event_control-rw-r--r-- 1 root root 0 Mar 10 09:50 cgroup.procs-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.failcnt--w- -- 1 root root 0 Mar 10 09:50 memory.force_empty-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.failcnt-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.limit_in_bytes-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.max_usage_in_bytes-r--r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.slabinfo- Rw-r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.tcp.failcnt-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.tcp.limit_in_bytes-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.tcp.max_usage_in_bytes-r--r--r-- 1 root root 0 Mar 10 09:50 memory.kmem.tcp.usage_in_bytes-r--r- -root root-1 Mar 10 09:50 memory.kmem.usage_in_bytes-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.limit_in_bytes-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.max_usage_in_bytes-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.move_charge_at_immigrate-r--r--r-- 1 root root 0 Mar 10 09:50 Memory.numa_stat-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.oom_control- 1 root root 0 Mar 10 09:50 memory.pressure_level-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.soft_limit_in_bytes-r--r--r-- 1 root root 0 Mar 10 09:50 memory.stat-rw-r--r-- 1 root root 0 Mar 10 09:50 memory .swappiness-r Mar Rafael-1 root root 0 Mar 10 09:50 memory.usage_in_bytes-rw-r--r-- 1 root root 0 Mar 10 09:50 memory.use_hierarchy-rw-r--r-- 1 root root 0 Mar 10 09:50 notify_on_release-rw-r--r-- 1 root root 0 Mar 10 09:50 tasks

The memory.limit_in_bytes file represents the total memory usage in byte.

Step 2: set the limit parameters. For example, here I want to limit the memory usage to 1G, then write 1073741824 to the memory.limit_in_bytes file as follows:

Root@cr7-ubuntu:~# echo 1073741824 > / sys/fs/cgroup/memory/mydocker/memory.limit_in_bytes

Step 3: add the process to the cgroup control group and write the current shell process ID to the tasks file:

Root@cr7-ubuntu:~# echo $$> / sys/fs/cgroup/memory/mydocker/tasks

Here we need to use the installation of the tool memtester,memtester. I won't go into details here.

After installing memtester, we execute the following command:

Root@cr7-ubuntu:~# memtester 1500m 1memtester version 4.2.2 (64-bit) Copyright (C) 2010 Charles Cazabon.Licensed under the GNU General Public License version 2 (only). Pagesize is 4096pagesizemask is 0xfffffffffffff000want 1500MB (1572864000 bytes) got 1500MB (1572864000 bytes), trying mlock. Killed

This command will request 1500 MB of memory and do a memory test. Since the memory limit for the current shell process is 1G above, cgroup kills memtester when the memory used by memtester reaches 1G.

The output of the last line above indicates that memtester wants 1500 megabytes of memory, but due to cgroup restrictions, it reaches the memory usage limit and is killed, as we expected.

We can use the following command to reduce the memory request and adjust the memory request to 500m:

Root@cr7-ubuntu:~# memtester 500m 1memtester version 4.2.2 (64-bit) Copyright (C) 2010 Charles Cazabon.Licensed under the GNU General Public License version 2 (only). Pagesize is 4096pagesizemask is 0xfffffffffffff000want 500MB (524288000 bytes) got 500MB (524288000 bytes) Trying mlock... locked.Loop 1/1: Stuck Address: ok Random Value: ok Compare XOR: ok Compare SUB: ok Compare MUL: ok Compare DIV: ok Compare OR: ok Compare AND: ok Sequential Increment: ok Solid Bits: ok Block Sequential: ok Checkerboard: ok Bit Spread: ok Bit Flip : ok Walking Ones: ok Walking Zeroes: ok 8-bit Writes: ok 16-bit Writes: okDone.

As you can see here, memtester has successfully applied for 500m of memory and completed the memory test normally.

Delete cgroups

If you don't want to use the cgroups created above, just delete the folder you created. For example, if I want to delete the mydocker directory in memory, use the following command:

Root@cr7-ubuntu:~# rmdir / sys/fs/cgroup/memory/mydocker/Docker uses cgroups to restrict container cpu

Limit the container to 20% of the CPU bandwidth.

Root@cr7-ubuntu:~# docker run-it-cpu-period=100000-cpu-quota=20000 ubuntu / bin/bash

After starting this container, we can confirm by looking at the contents of the resource limit file in the control group "docker" in the CPU subsystem under the Cgroups file system:

Root@cr7-ubuntu:~# cat / sys/fs/cgroup/cpu/docker/99cdb60e28191871c456f736470c9a1d344b95c6bcdb39036926428378db55c4/cpu.cfs_period_us100000root@cr7-ubuntu:~# cat / sys/fs/cgroup/cpu/docker/99cdb60e28191871c456f736470c9a1d344b95c6bcdb39036926428378db55c4/cpu.cfs_quota_us20000

The very long number + letter is the id of the container, and you can use the docker inspect command to view the id of the container.

Limit the container memoryroot@cr7-ubuntu:~# docker run-it-masks 1g nginx

The above command creates and starts a nginx container and limits the memory to 1G. Then we look at the contents of the memory.limit_in_bytes file of the cgroup control group corresponding to the container, and we can see that the memory limit is exactly 1G:

Root@cr7-ubuntu:~# cat / sys/fs/cgroup/memory/docker/2e5f7db4b9a08b08471b3fcf9f71cb14396fb081db3cdc25140714ae037f2136/memory.limit_in_bytes1073741824 at this point, the study on "how to achieve resource restrictions through the Cgroups mechanism" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.