Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to use cgroup's cpuset controller to limit the CPU use of the process", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let Xiaobian take you to learn "how to use cgroup's cpuset controller to limit the CPU use of the process"!

Recently, I have been doing some performance testing, and the first premise is to control the CPU usage. The most intuitive approach is undoubtedly to install Docker and run the benchmark in each container configured with parameters.

For computation-intensive tasks, it is undoubtedly the most convenient to limit CPU usage directly with Linux's native cgroup function under the requirement of limiting CPU only.

This article briefly explains how to use cgroup's cpuset controller to restrict processes to using only a few CPUs, or more precisely, a few logical cores.

1. View CPU configuration

Common configurations can be viewed with the following BASH command.

1

2

3

cat /proc/cpuinfo | grep "physical id" | sort |uniq #View number of physical CPUs

cat /proc/cpuinfo | grep "cores" |uniq #View the number of cores per CPU

cat /proc/cpuinfo | grep "processor" |wc -l #View the total number of logical threads on the host

Specifically, with hyperthreading enabled, each CPU physical core emulates two threads, also known as logical cores. The judgment method is as follows:

1

Hyperthreading enabled = Number of physical CPUs * Number of cores per CPU/Total number of logical threads == 2

2. What is NUMA?

Here mentioned a concept called NUMA, if there are multiple CPU blocks inserted on the motherboard, then it is NUMA architecture. Each CPU has an exclusive area and generally has an independent fan.

A NUMA node contains hardware devices such as CPU and memory directly connected to the area, and the communication bus is generally PCI-E. This also introduces the concept of CPU affinity, where a CPU accesses memory on the same NUMA node faster than another node.

Execute the following command to view the NUMA structure of the native machine.

1

2

3

4

5

6

7

8

9

10

11

12

numactl --harware

# available: 2 nodes (0-1)

# node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# node 0 size: 127834 MB

# node 0 free: 72415 MB

# node 1 cpus: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

# node 1 size: 128990 MB

# node 1 free: 68208 MB

# node distances:

# node 0 1

# 0: 10 21

# 1: 21 10

A NUMA node consists of a memory node and several logical cores belonging to the same CPU block, remember their numbers, which will be used in configuring cpuset.

In this interpretation of "node distance," the communication cost of accessing memory on this node is a constant value of 10, which is used by the operating system to quantify the cost of accessing memory on other NUMA nodes.

3. Create cgroups and configure resource usage

Linux distributions with higher kernel versions (>=2.6.24) have cgroups built in, which can be verified by executing the following command:

1

2

3

4

5

6

7

8

9

10

11

12

mount | grep cgroup

# tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)

# cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)

# cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)

# cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)

# cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)

# cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)

# cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)

# cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)

# cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)

# cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)

# cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)

If not, you can also execute a simple script to mount it all at once.

1

2

3

4

5

6

7

cgroot="${1:-/sys/fs/cgroup}"

subsys="${2:-blkio cpu cpuacct cpuset devices freezer memory net_cls net_prio ns perf_event}"

mount -t tmpfs cgroup_root "${cgroot}"

for ss in $subsys; do

mkdir -p "$cgroot/$ss"

mount -t cgroup -o "$ss" "$ss" "$cgroot/$ss"

done

For each resource, cgroup provides a controller to configure, which is represented in Linux as a file system. This article deals only with the placement of processes on the physical core, so take a look at what's in the cpuset directory.

1

2

3

4

5

6

7

8

9

10

11

mkdir /sys/fs/cgroup/cpuset/tiger #Create a control group and delete it with rmdir command

ls /sys/fs/cgroup/cpuset/tiger

# cgroup.clone_children cpuset.memory_pressure

# cgroup.procs cpuset.memory_spread_page

# cpuset.cpu_exclusive cpuset.memory_spread_slab

# cpuset.cpus cpuset.mems

# cpuset.effective_cpus cpuset.sched_load_balance

# cpuset.effective_mems cpuset.sched_relax_domain_level

# cpuset.mem_exclusive notify_on_release

# cpuset.mem_hardwall tasks

# cpuset.memory_migrate

If you want to use cpuset controller, you need to configure both cpuset.cpus and cpuset.mems files (parameters). These two files accept intervals denoted by dashes and commas, such as "0- 7, 16 -23." If the corresponding resource does not exist, an error will be reported when writing.

It is not recommended to configure directly under the root directory of the controller. Multiple controllers can be maintained simultaneously by creating subdirectories. Execute the following command to restrict all processes in the tiger control group to logical cores 0 and 1.

1

2

echo "0-1" > /sys/fs/cgroup/cpuset/tiger/cpuset.cpus

echo 0 > /sys/fs/cgroup/cpuset/tiger/cpuset.mems

For the cpuset.mems parameter, each memory node corresponds to a NUMA node. If the memory requirements of the process are large, all NUMA nodes can be configured. This is where the NUMA concept comes in. For performance reasons, logical cores and memory nodes configured generally belong to the same NUMA node, and their mapping can be learned with the "numactl --hardware" command.

4. verify the effect

Among all configuration files for cpuset, tasks and cgroups.procs are used to manage processes in control groups. Execute the following command to add the current session to the control group you just created. All commands (child processes) initiated by this session will receive cpu constraints.

1

echo $$ > /sys/fs/cgroup/cpuset/tiger/cgroup.procs #Write the current process number

The two configuration items are basically equivalent, but there is a small difference. The operating system uses threads as scheduling units and writes a general pid to tasks. Only the thread corresponding to this pid, as well as other processes and threads generated by it, will belong to this control group. By writing pid to cgroups.procs, the operating system adds all threads that find their own processes to the current control group.

When a process joins a control group, the restrictions associated with the control group take effect immediately. Start a computationally intensive task that requires 4 logical cores.

1

2

3

stress -c 4 &

# [1] 2958521

# stress: info: [2958521] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd

Observe CPU usage to verify the effect, only the two logic cores numbered 0 and 1 are working, and the proportion of user mode is as high as 100%.

1

2

3

4

5

6

top #Press the number 1 key on the list page to switch to CPU Kanban

# Tasks: 706 total, 3 running, 702 sleeping, 0 stopped, 1 zombie

# %Cpu0 :100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

# %Cpu1 :100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

# %Cpu2 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

# %Cpu3 : 0.0 us, 1.7 sy, 0.0 ni, 98.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

At this point, I believe that everyone has a deeper understanding of "how to use cgroup's cpuset controller to limit the CPU use of the process," and may wish to actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.