Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to understand system load in Linux". In daily operation, I believe many people have doubts about how to understand system load in Linux. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubt of "how to understand system load in Linux"! Next, please follow the editor to study!

Generally speaking, on unix-like systems, there is a system load (load average), which is used to describe the busy degree of the system. The higher the value is, the busier the system is.

View load

$uptime19:59:57 up 29 days, 7:08, 1 user, load average: 0.57,0.26,0.18

We focus on the three values after load average, which represent the average system load of 1 minute, 5 minutes and 15 minutes, respectively. If the value of 1 minute > 5 minutes > 15 minutes, it means that the system pressure is increasing in the last 15 minutes, and vice versa.

Similarly, in the first line of the top command, you can see the system load, which has the same meaning as uptime.

What is the load?

Generally speaking, system threads are basically in these three states: running, runnable, blocking and waiting, in which the running thread is running on CPU, the runnable thread is waiting for CPU scheduling, and the blocked thread is waiting for lock release or io completion.

On traditional unix systems (such as BSD), the system load consists of running threads and runnable threads.

It can well explain the saturation of CPU, such as 4-core CPU. If the load is always higher than 4, it means that CPU resources are saturated.

Linux expands the definition of load as follows:

The Linux load consists of three parts: running threads and runnable threads, and D-state threads (usually waiting for io to complete).

Thread status D

In Linux, threads have the following common states:

R: running or runnable state

D: uninterruptible sleep state, usually waiting for io to complete

The threads in R and D state will affect the system load, so when the system load is high, you can know which threads are caused by the following command:

Ps-eLo pid,tid,stat,comm | grep-E "R | D"

Small experiment: increase the system load to 100

# create a child process using the vfork function. If the child process does not call the exec system call, its state will always be D.

$cat uninterruptible.c int main () {vfork (); sleep; return 0;} # compiled into an executable program $gcc-o uninterruptible uninterruptible.c# runs 100 programs $for i in {1... 100}; do. / uninterruptible &; done

Wait 1 minute and you will find that the system load has risen to nearly 100, as follows:

$uptime20:24:42 up 29 days, 7:32, 1 user, load average: 99.94,74.82,35.8 processes with D status can be seen $ps-eLo pid,tid,stat,pcpu,wchan:32 Comm | grep "D" 3774195 3774195 D 0.0 do_fork uninterruptible3774196 3774196 D 0.0 do_fork uninterruptible3774197 3774197 D 0.0 do_fork uninterruptible3774198 3774198 D 0.0 do_fork uninterruptible

As shown above, you can see the thread status through the ps command, and there is a wchan field that shows what kernel function the thread is currently blocked on, which shows some clues.

In addition, you can see the code path when the D thread is blocked through / proc/sysrq-trigger, as follows:

# write a w, need root permission to execute $echo w > / proc/sysrq-trigger#, and then the kernel will output the D-state thread call stack to the kernel log, which can be viewed through dmesg $dmesg

It is clear here that this is due to the increased load caused by vfork system calls.

We have previously introduced the offcputime tool in the bcc toolset, which can be used to draw offcpu flame diagrams. Similarly, when diagnosing high load problems, you can also use this tool to pass a parameter that only focuses on the offcpu behavior of the D-status thread, as follows:

# ubuntu installation bcc toolset $sudo apt install bpfcc-tools# enters bash$ sudo bash# using root identity-- state 2 is used to specify the offcpu stack $offcputime-bpfcc-K-- state 2-f 60 > d_state_offcpu_stack.out# for crawling TASK_UNINTERRUPTIBLE, that is, D status thread, and drawn as offcpu flame graph $awk'{print $1, $2 / 1000} 'd_state_offcpu_stack.out |. / FlameGraph/flamegraph.pl-- color=io-- countname=ms > d_state_offcpu.svg

At this point, the study on "how to understand the system load in Linux" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.