Shulou(Shulou.com)06/01 Report--

Today, I would like to share with you the relevant knowledge points of the commonly used Linux performance monitoring tools, which are detailed in content and clear in logic. I believe most people still know too much about this knowledge, so share this article for your reference. I hope you can get something after reading this article.

1. Uptime$ uptime 23:51:26 up 21:31, 1 user, load average: 30.02, 26.43, 19.0212

This command can roughly see the overall load of the computer, and the numbers after load average represent the average load of the computer in 1min, 5min and 15min, respectively.

2. Dmesg | tail$ dmesg | tail [1880957.563150] perl invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0 [...] [1880957.563400] Out of memory: Kill process 18694 (perl) score 246 or sacrifice child [1880957.563408] Killed process 18694 (perl) total-vm:1972392kB, anon-rss:1953348kB, file-rss:0kB [2320864.954447] TCP: Possible SYN flooding on port 7001. Dropping request. Check SNMP counters.123456

Print the contents of the kernel ring cache, which can be used to view some errors

In the above example, process 18694 is dropped by kill and TCP request is discarded because memory is out of bounds. Through dmesg, you can quickly determine whether there are any problems that lead to abnormal system performance.

3. Vmstat 1$ vmstat 1procs-memory--swap---io-----system---cpu- r b swpd free buff cache si so bi bo in cs us sy id wa st34 00 200889792 73708 591828 00 00 0 5 6 10 96 1 3 032 00 200889920 73708 591860 0 00 592 13284 4282 98 11 032 00 200890112 73708 591860 00 9501 2154 99 1 00 032 00 200889568 73712 591856 00 048 11900 2459 99 00 032 00 200890208 73712 591860 00 15898 4840 98 11 00 ^ C123456789

Print statistics for processes, memory, swap partitions, IO, CPU, etc.

The format of vmstat is as follows

> vmstat [options] [delay [count]]

The first output of vmstat represents the average value from boot to vmstat runtime; the rest of the output is the average within the specified time interval. In the above example, the value of delay is set to 1, except for the first time, the rest is counted once in a second. If count is not set, it will be printed in a loop.

$vmstat 10 3procs-memory--swap---io-----system---cpu- r b swpd free buff cache si so bi bo in cs us sy id wa st 100 2527112 1086888 13720228 0 1 14 2 11 1 99 0 2527112 1086888 13719856 00 0 104 3003 4901 00 990 00 00 2526412 1086888 13719904 00 0 10 3345 4870 01 990 0123456

In the above example, delay is set to 10 and the count is set to 3, which means that each line is printed for an average of 10 seconds, only 3 times.

Columns to be checked

R: indicates the number of processes running or waiting for CPU scheduling. Because this column of data does not contain statistics for Ibank O, it can be used to detect whether the CPU is saturated. If the number in the r column is greater than the core number of the CPU, it means that the CPU has been saturated.

Free: current remaining memory

Si, so: the number of swap partitions swapped in and out. If the number of swapping in and out is greater than 0, there is insufficient memory.

The statistical information of us, sy, id and wa:CPU, which represents user time, system time (kernel), idle, and wait I Zero, respectively. The time it takes to process system time O is included in the system time, so if the system time exceeds 20%, there may be bottlenecks or anomalies in it.

4. Mpstat-P ALL 1$ mpstat-P ALLLinux 3.10.0-229.el7.x86_64 (localhost.localdomain) 05Accord 2018 _ x86 December 6416 CPU 04:03:55 PM CPU% usr% nice% sys% iowait% irq% soft% steal% guest% gnice% idle04:03:55 PM all 3.67 0.00 0.61 0.71 0.00 0.00. 00 0.00 0.00 95.0204:03:55 PM 0 3.52 0.00 0.57 0.76 0.00 0.00 0.00 95.1504:03:55 PM 1 3.83 0.00 0.61 0.71 0.00 0.00 0.00 94.8504:03:55 PM 2 3.80 0 . 00 0.61 0.60 0.00 0.00 0.00 94.9904:03:55 PM 3 3.68 0.00 0.58 0.60 0.00 0.00 0.00 95.1304:03:55 PM 4 3.54 0.00 0.57 0.60 0.00 0.00 0.00 0.00 95.30 [...] 1234567891011

This command is used to print statistics for each CPU once a second and can be used to see if the scheduling of the CPU is uniform.

5. Pidstat 1$ pidstat 1Linux 3.13.0-49-generic (titanclusters-xxxxx) 07 rcuos/007:41:03 PM 14 CPU 07:41:02 PM UID PID% usr% system% guest% CPU CPU Command07:41:03 PM 0 9 0.00 0.94 1 rcuos/007:41:03 PM 0 4214 5.66 5. 66 0.00 11.32 15 mesos-slave07:41:03 PM 0 4354 0.94 0.94 0.00 1.89 8 java07:41:03 PM 0 6521 1596.23 1.89 0.00 1598.11 27 java07:41:03 PM 0 6564 1571.70 7.55 0.00 1579.25 28 java07:41:03 PM 60004 60154 0.94 4.72 0 .00 5.66 9 pidstat07:41:03 PM UID PID% usr% system% guest% CPU CPU Command07:41:04 PM 0 4214 6.00 2.00 8.00 15 mesos-slave07:41:04 PM 0 6521 1590.00 1.00 0.00 1591.00 27 java07:41:04 PM 0 6564 1573.00 10.00 1583 .00 28 java07:41:04 PM 108 6718 1.00 0.00 0.00 1.00 snmp-pass07:41:04 PM 60004 60154 1.00 4.00 0.00 5.00 9 pidstatC123456789101112131415161718

This command is used to print the CPU usage of each process, similar to what is shown in the top command. The advantage of pidstat is that it scrolls the operation of the process, rather than clearing the screen as top does.

In the above example, the cpu utilization of the two java processes in% CPU reached 1590% and 1573%, respectively, indicating that the java process occupied 16 CPU.

6. Iostat-xz 1

Similar to vmstat, the first output is the sampling data from the time when the system is powered on to the time of statistics.

$iostat-xz 1Linux 3.13.0-49-generic (titanclusters-xxxxx) 07 steal 14 avg-cpu:% user% nice% system% iowait% steal% idle 73.96 0.00 3.73 0.03 0.06 22.21Device: rrqm/s wrqm/s rmer s Wamp s rkB/s wkB/s avgrq-sz avgqu-sz await r _ Await w_await svctm% utilxvda 0.00 0.23 0.21 0.18 4.52 2.08 34.37 0.00 9.98 13.80 5.42 2.44 0.09xvdb 0.01 0.00 1.02 8.94 127.97 598.53 0.43 1.78 0.28 0.2500 .25xvdc 0.01 0.00 1.02 8.86 127.79 595.94 146.50 0.00 0.45 1.82 0.30 0.27 0.26dm-0 0.00 0.00 0.69 2.32 10.47 31.69 28.01 0.01 3.23 0.71 3.98 0.13 0.04dm-1 0.00 0.00 0.94 0.01 3.78 8.00 0.33 345.84 0.04 346.81 0.01 0.00dm-2 0.00 0.09 0.07 1.35 0.36 22.50 0.00 2.55 0.23 5.62 1.78 0.03 [...] ^ C123456789101112131415

Check column

The number of reads, writes, read Kbytes, and write Kbytes sent to the reads O device per second is indicated by r _ hand s, w _ hand s, rkB/s, wkB/s.

Await, which represents the average Imax O time that the application was queued and serviced. If this value is greater than the expected time, it indicates that the IActionO device is saturated or abnormal.

Avgqu-sz, which represents the average time that the request was sent to the Igamot device. If the value is greater than 1, the Icano device may be saturated.

% util, the utilization rate of the device per second; if the utilization rate exceeds 60%, the device has abnormal performance

7. Free-m $free-m total used free shared buffers cachedMem: 245998 24545 221453 83 59 541 buffers/cache: 23944 222053Swap: 0 012345

Columns checked:

Buffers: For the buffer cache, used for block device I/O.cached: For the page cache, used by file systems.

If buffers and cached are close to 0, it means that the utilization rate of Icano is too high, and there are performance problems in the system. Free memory is used as cache in Linux. If the application needs to allocate memory, the system can quickly reclaim the memory occupied by cache, so the memory of free contains the part occupied by cache.

8. Sar-n DEV 1

Sar is an acronym for System Activity Reporter, system activity status report.

-n keyword [,...] ALL, used to report network statistics. Keyword can be one or more of the following: DEV, EDEV, NFS, NFSD, SOCK, IP, EIP, ICMP, EICMP, TCP, ETCP, UDP, SOCK6, IP6, EIP6, ICMP6, EICMP6 and UDP6.

-n DEV 1, count the network usage once per second;-n EDEV 1, count the wrong network information once per second

$sar-n DEV 1Linux 3.10.0-229.el7.x86_64 (localhost.localdomain) 05Action31Universe 2018 _ x86milli6416 CPU 03:54:57 PM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s03:54:58 PM ens32 3286.00 7207.00 283.34 18333.90 0.00 0.0003 localhost.localdomain 54: 58 PM lo 0.00 0.00 0.00 0.0003:54:58 PM vethe915e51 0.00 0.00 0.00 0.0003:54:58 PM docker0 0.00 0.00 0.00 0.0003:54:58 PM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s03:54:59 PM ens32 3304.00 7362.00 276.89 18898.51 0.00 0.00 0.0003:54:59 PM lo 0.00 0.00 0.00 0.000. 0003Viru 54 PM vethe915e51 59 PM docker0 0.00 0.00 0.00 0.0003 VLV 54 59 0.00 0.00 0.00 ^ C123456789101112131415IFACE Network interface name Rxpck/s, packets received per second; txpck/s, packets transmitted per second; (transmit packages) rxkB/s, kilobytes per second received; txkB/s, kilobytes sent per second; rxcmp/s, number of compressed packets received per second; txcmp/s, number of compressed packets sent per second; rxmcst/s, number of group packets received per second; 9. Sar-n TCP,ETCP 1

This command can be used to roughly determine the throughput of the network, such as the number of network connections initiated and the number of network connections received

TCP, reporting statistics on TCPv4 network traffic

ETCP, reporting statistics about TCPv4 network errors

$sar-n TCP ETCP 1Linux 3.10.0-514.26.2.el7.x86_64 (aushop) 05 CPU 2018 _ x86 pound 642 CPU 04:16:27 PM active/s passive/s iseg/s oseg/s04:16:44 PM 0.00 2.00 15.00 13.0004 ETCP 1Linux 45 PM 0.00 3.00 126.00 203.0004 ETCP 1Linux 16 0.00 0.00 99.00 99.0004:16:47 PM 0.00 0.00 18.00 9.0004:16:48 PM 0.00 0.00 5.00 6.0004:16:49 PM 0.00 0.00 1.00 1.0004:16:50 PM 0.00 1.00 4.00 4.0004:16:51 PM 0.00 3.00 171.00 243.00 ^ C12345678910111213

Detected columns:

Active/s: Number of locally-initiated TCP connections per second (e.g., via connect ()), number of network connections initiated; passive/s: Number of remotely-initiated TCP connections per second (e.g., via accept ()), number of network connections received; retrans/s: Number of TCP retransmits per second, number of retransmissions; 10. Top

The top command contains more metric statistics, which is equivalent to a comprehensive command.

Toptop-00:15:40 up 21:56, 1 user, load average: 31.09,29.87, 29.92Tasks: 871 total, 1 running, 868 sleeping, 0 stopped, 2 zombie%Cpu (s): 96.8 us, 0.4 sy, 0.0 ni, 2.7 id, 0.1 wa, 0.0 hi, 0.0 si, 0.0 stKiB Mem: 25190241+total, 24921688 used, 22698073+free, 60448 buffersKiB Swap: 0 total, 0 used 0 free. 554208 cached Mem PID USER PR NI VIRT RES SHR S% CPU% MEM TIME+ COMMAND 20248 root 20 0 0.227t 0.012t 18748 S 3090 5.229812 java 4213 root 20 2722544 64640 44232 S 23.5 233R1.00.0 root 00.07 top 5235 root 20 038.227g 54700449996 S 0.70.2 2:02.74 java 4299 root 20 0 20.015g 2.682g 16836 S 0.3 1.1 33:14.42 java 1 root 20 0 33620 2920 1496 S 0.0 0.0 0:03.82 init 2 root 20 00 00 S 0.0 0.0 0:00.02 kthreadd 3 root 20 00 00 S 0.0 0.00: 05.35 ksoftirqd/0 5 root 0-20 000 S 0.0 0.00: 00.00 kworker/0:0H 6 root 20 00 00 S 0.0 0.00: 06.94 kworker/u256:0 8 root 20 00 S 0.0 2 root 38.05 rcu_sched1234567891011121314151617181911. Summary

The following picture shows the main functions of various commands, such as using vmstat to view the overall performance of the system, mpstat to view the performance of cpu, pidstat to view the status of processes, iostat to view the status of io, free to view the status of memory, sar to view the status of the network, and so on. Image.png

An overview of common performance tools for Linux

These are all the contents of this article entitled "what are the commonly used Linux performance monitoring tools?" Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.