Shulou(Shulou.com)06/01 Report--

This article will explain in detail the example analysis of the CentOS service program performance evaluation document. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

1 Overview

1.1 factors affecting the performance of Linux service programs

CPU, memory, disk Imax O bandwidth, network Imax O bandwidth

1.2 performance evaluation

CPU:user% + sys%

< 70%； 程序在用户态和内核态的执行时间百分比。 内存：Swap In（si）＝0； Swap Out（so）＝0； 以不使用交换分区为准，如果频繁用到交换分区，内存可能不够了。 硬盘：iowait % < 20%； 网络：只要有足够带宽尽情的使用吧，达到网卡带宽linux系统表示毫无压力 其中： %user：表示CPU处在用户模式下的时间百分比。 %sys：表示CPU处在内核模式下的时间百分比。 %iowait：表示CPU等待输入输出完成时间的百分比。 swap in：即si，表示虚拟内存的页导入，即从SWAP DISK交换到RAM swap out：即so，表示虚拟内存的页导出，即从RAM交换到SWAP DISK。 1.3性能分析工具 常用系统命令：top、free、ps、uptime、iotop、vmstat、iostat 、dstat、sar。 使用方法：top命令把握全局，使用特定命令深入分析 常用组合方式： （1）CPU瓶颈：top、vmstat、iostat、sar –u、sar -q （2）内存瓶颈：free、vmstat、sar -B、sar –r、sar -W （3）磁盘I/O瓶颈：iotop、iostat、sar -b、sar –u、sar -d （4）网络瓶颈： dstat 2 top 2.1功能 提供了实时的对系统处理器、内存、任务等状态监视；该命令可以按CPU使用、内存使用对任务进行排序；TOP是一个动态显示过程，可以通过用户按键来不断刷新当前状态，也可以在启动时指定刷新间隔。 2.2命令输出示意图 top - 10:16:29 up 38 days, 15:48, 5 users, load average: 0.04, 0.10, 0.05 Tasks: 569 total, 2 running, 562 sleeping, 0 stopped, 5 zombie Cpu(s): 2.6%us, 1.3%sy, 0.4%ni, 95.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 3839112k total, 3151560k used, 687552k free, 302944k buffers Swap: 6078456k total, 631852k used, 5446604k free, 348548k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10603 zhixiang 20 0 1225m 284m 30m S 4.6 7.6 271:48.24 vs_exe 2473 xulun 20 0 118m 43m 6932 S 3.0 1.2 964:14.99 Xvnc 1412 xulun 30 10 232m 12m 5216 S 2.3 0.3 0:04.59 floaters 14823 chujie 20 0 1112m 247m 17m S 2.0 6.6 363:51.38 vs_exe 17772 sihao 20 0 1101m 202m 10m S 1.7 5.4 631:21.17 vs_exe 11054 sihao 20 0 906m 65m 9412 S 1.0 1.7 410:06.55 vs_exe 20782 yongtao 20 0 1049m 173m 9996 S 1.0 4.6 196:05.10 vs_exe 14637 chujie 20 0 1274m 132m 2388 S 0.7 3.5 63:20.18 KugooPopMsgServ 1439 yanyun 20 0 15428 1600 940 R 0.3 0.0 0:00.01 top 3491 zhixiang 20 0 129m 55m 17m S 0.3 1.5 10:57.25 Xvnc 1 root 20 0 19344 1200 984 S 0.0 0.0 0:00.89 init 2.3命令输出解析 Top输出的信息很多基本囊括了所有的性能指标，前五行是统计信息区，表示的是系统整体的统计信息，包括：系统负载、任务、CPU、内存等；后面的是每个进程相关信息。 第一行是任务队列信息，同uptime命令的执行结果是一样的 [yanyun@~/test]$ uptime 10:20:17 up 38 days, 15:52, 5 users, load average: 0.00, 0.04, 0.03 其内容如下：10:20:17：当前时间 38 days, 15:52：系统运行时间 5 users：当前登录用户数 load average: 0.00, 0.04, 0.03：系统负载，即任务队列的平均长度。三个数值分别为 1分钟、5分钟、15分钟前到现在的平均值。 第二行为进程的信息 内容如下：569 total：进程总数 2 running,：正在运行的进程数 562 sleeping：睡眠的进程数 0 stopped：停止的进程数 5 zombie：僵尸进程数 第三行为CPU的信息 内容如下：2.6%us：用户空间占用CPU百分比 1.3%sy：内核空间占用CPU百分比 0.4%ni：用户进程空间内改变过优先级的进程占用CPU百分比 95.7%id：空闲CPU百分比 0.0%wa：等待输入输出的CPU时间百分比 第四、五行为内存信息；命令输出跟free相同 [yanyun@~]$ free total used free shared buffers cached Mem: 3839112 3256976 582136 0 143664 444992 -/+ buffers/cache: 2668320 1170792 Swap: 6078456 574772 5503684 内容如下：Mem :3839112k total：物理内存总量 3151560k used：使用的物理内存总量 687552k free：空闲内存总量 302944k buffers：用作缓冲的内存量 Swap: 6078456k total：交换区总量 631852k used：使用的交换区总量 5446604k free：空闲交换区总量 348548k cached：缓存总量。 注：buffer：可以认为是写出磁盘的缓冲区； Cache：读出磁盘的缓存。 Linux系统使用内存的原则是：不用白不用，用了也白用；尽可能的缓存东西，所以往往看空闲内存很小，但是cache很大；Linux系统会定时启动内核线程kswapd进行缓存回收。 后面的是显示每个进程相关信息 %CPU：上次更新到现在的CPU时间占用百分比 TIME+：进程使用的CPU时间总计 %MEM：进程使用的物理内存百分比 VIRT：进程使用的虚拟内存总量，单位kb RES：进程使用的、未被换出的物理内存大小，单位kb。RES=CODE+DATA SHR：共享内存大小，单位kb S：进程状态。(D=不可中断的睡眠状态 R=运行 S=睡眠 T=跟踪/停止 Z=僵尸进程) 2.4常用选项 top [-] [d][p][M][P] 参数说明： d：指定每两次屏幕信息刷新之间的时间间隔。(top –d 1：每秒刷新一次) p：指定进程ID来仅监控某个进程。(top –d 1234：只查看pid为1234的进程信息) k：终止一个进程。Top运行时参数，系统将提示用户输入需要终止的进程PID，以及需要发送给该进程什么样的信号。使用信号9强制结束该进程。 M：根据驻留内存大小进行排序。 P：根据CPU使用百分比大小进行排序。 注：在命令行执行过程中按下数字键‘1’，可以查看到CPU每个核的相关信息。 Tasks: 564 total, 3 running, 556 sleeping, 0 stopped, 5 zombie Cpu0 : 2.9%us, 2.9%sy, 0.0%ni, 94.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 2.0%us, 2.0%sy, 0.0%ni, 96.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 3839112k total, 3445296k used, 393816k free, 48180k buffers Swap: 6078456k total, 553876k used, 5524580k free, 976128k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 14823 chujie 20 0 1112m 245m 16m R 3.0 6.5 373:07.82 vs_exe 5589 sihao 20 0 1019m 267m 29m R 2.0 7.1 2:24.80 vs_exe 5674 zhixiang 20 0 1103m 253m 37m S 2.0 6.8 4:17.89 vs_exe 3 iotop： 3.1功能 是一个用来监视每个线程的磁盘 I/O 使用状况的类top 工具， 注：此命令需要自行安装(yum install iotop) 3.2命令输出示意图 Total DISK READ: 50.23 M/s | Total DISK WRITE: 34.25 K/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO>

COMMAND

61524 be/4 root 47.65M be/4 root s 0.00 Bhand s 0.00% 37.83%. / relay_server

61539 be/4 root 121.77 Kswab s 0.00 Bhand s 0.00% 26.73%. / relay_server

61544 be/4 root 700.15 Kswab s 0.00 Bhand s 0.00% 24.89%. / relay_server

61543 be/4 root 528.92 KUnip s 0.00 Bhand s 0.00% 21.29%. / relay_server

61541 be/4 root 494.67 KUnip s 0.00 Bhand s 0.00% 21.22%. / relay_server

61540 be/4 root 323.44 Kswab s 0.00 Bhand s 0.00% 8.62%. / relay_server

61542 be/4 root 468.04 Kswab s 0.00 Bhand s 0.00% 8.13%. / relay_server

480 be/3 root 0.00 BUnip s 0.00 BUnip s 0.00% 0.02% [jbd2/sda2-8]

1 be/4 root 0.00 BUnip s 0.00 BUnip s 0.00% 0.00% init

2 be/4 root 0.00 BUnip s 0.00 BUnip s 0.00% 0.00% [kthreadd]

3 rt/4 root 0.00 BUnip s 0.00 BUnip s 0.00% 0.00% [migration/0]

3.3 Command output parsing

First line:

Total DISK READ: 50.23MUniverse: the amount of data read by the disk per second

Total DISK WRITE: 34.25Kamp s: the amount of data written to the disk per second

The following is the IO of each thread:

It's relatively simple; just explain IO: it has the same meaning as the wa of top, but the wa of a thread represented here.

3.4 Common options

Iotop [-] [d] [p]

Parameter description:

D: specifies the time interval between every two screen information refreshes. (iotop-d 1: refresh once per second)

P: specify thread ID to monitor only one thread. (iotop-d 1234: view only process information with a pid of 1234)

Note: by pressing the letter'IO 'during command line execution, you can view only threads with the command line.

Pstree-p: view the process tree to output the relationships between processes.

Ps-eLf: view threads. Ps-Premium version of ef

4 vmstat:

4.1 Featur

Tools to view virtual memory (Virtual Memory) usage

4.2 Command output schematic diagram

[yanyun@~/test] $vmstat

Procs-memory--swap---io-----system---cpu-

R b swpd free buff cache si so bi bo in cs us sy id wa st

0 0 631376 834896 15108 452024 0 0 4 4 1 2 2 1 97 0 0

4.3 Command output parsing

Memory: the fourth and fifth elements of the same message as top.

Swap: memory to swap partition swap in and out rate.

Io: read and write disk rat

System in: interrupts per second, including clock interrupts

System cs: the number of environment (context) switches per second; frequent switching has an impact on system performance. Linux provides thread affinity to cpu and binds a thread to a core run.

Cpu: the third line of the same output as top

Note: the swap entry is not zero, all swap partitions are used, and memory may have been used up and become a bottleneck of the system.

4.4 Common options

Vmstat 1 10: refreshes once per second for a total of ten exits.

Note: the following options are professional to view a specific item of information, there are a few output a lot of information, then use google!

-a: displays active and inactive memory

-f: displays the number of fork since the system was started

-m: display slabinfo

-s: displays memory-related statistics and a variety of system activity.

-d: displays disk-related statistics.

-p: displays the specified disk partition statistics

5 iostat

5.1 function

Used to output statistics related to CPU and disk Imaco

5.2 schematic diagram of output result

[root@ShanWei_119_134_255_208 ~] # iostat-x

Linux 2.6.32-279.el6.x86_64 (ShanWei_119_134_255_208) 07 ShanWei_119_134_255_208 30 CPU 2013 _ x86 pound 6416)

Avg-cpu:% user nice% system% iowait% steal% idle

2.63 0.00 2.25 5.98 0.00 89.14

Device: rrqm/s wrqm/s rUnip s wdeband s rsec/s wsec/s avgrq-sz avgqu-sz await svctm% util

Sda 195.42 163.72 194.33 126.28 28464.35 2319.94 96.02 1.16 3.63 1.41 45.30

Sdb 0.33 48.97 122.71 2.70 29603.37 413.38 239.34 0.27 2.15 1.00 12.52

5.3 Command output interpretation

Avg-cpu segment:

% user: the percentage of CPU used to run at the user level.

% nice: the percentage of CPU used by the nice operation.

% sys: the percentage of CPU used to run at the system level (kernel).

% iowait: percentage of CPU occupied when CPU waits for hardware Imax O.

% idle: percentage of CPU idle time.

Device segment:

Tps: the number of iCompO requests sent per second

Blk_read / s: number of block read per second

Blk_wrtn/s: number of block written per second

Blk_read: total number of block read

Blk_wrtn: total number of block written

The-x option can view more information as follows

Rrqm/s: how many read requests related to this device are Merge per second

Wrqm/s: how many write requests related to this device are Merge per second

Rsec/s: number of sectors read per second

Number of sectors written by wsec/: per second

Avgrq-sz: average data size (sector) per device Istroke O operation

Avgqu-sz: average Imax O queue length

Await: average wait time (in milliseconds) for each device Istroke O operation

Svctm: average service time per device Istroke O operation (in milliseconds)

% util: all processing IO times during the statistical time, so this parameter indicates how busy the device is

5.4 Common options

Iostat-d 26: displays device statistics every 2 seconds. A total of 6 output times.

-x: check the device utilization and response time

6 dstat

6.1 featur

Is an omnipotent system information statistics tool, which can only monitor the whole system but not analyze a process or program in depth; the monitoring items include: cpu, disk, memory, network card, process, system (color interface linux is rare!)

Note: this command needs to be installed by yourself (yum install dstat)

6.2 Command output schematic diagram

[root@ShanWei_119_134_255_208 ~] # dstat-cdlmnpy

-total-cpu-usage-----dsk/total- load-avg----memory-usage--net/total- procs--- system--

Usr sys idl wai hiq siq | read writ | 1m 5m 15m | used buff cach free | recv send | run blk new | int csw

3 2 89 600 | 28m 1367k | 5.41 5.07 4.87 | 4019m 172m 58.5g 267m | 0 | 0.0.0 1.3 | 8608 35k

21 85 12 00 | 21m 828k | 5.41 5.07 4.87 | 4020m 172m 58.6g 243m | 191k 15m | 06.00 | 5711 18k

1 1 80 18 00 | 5828k 4804k | 5.41 5.07 4.87 | 4018m 172m 58.6G 260m | 190k 17m | 2.03.00 | 5802 12k

6.3 Command output interpretation

It has basically been explained before.

6.4 Common options

Dstat-cdlmnpsy:cdlmnpsy these options basically include the commonly used, of course, there are a lot of dstat-help it.

7 sar

7.1 featur

System Activity Reporter system activity report is one of the most comprehensive system performance analysis tools on Linux at present. It can report system activities from many aspects, including: reading and writing of files, usage of system calls, disk I / O, CPU efficiency, memory usage, process activities and IPC-related activities.

7.2 Command output schematic diagram

[root@localhost] # sar-r 1 100

Linux 2.6.32-220.el6.x86_64 (localhost.localdomain) 07 localhost.localdomain 30 CPU 2013 _ x86 pound 648)

03:17:59 PM kbmemfree kbmemused memused kbbuffers kbcached kbcommit commit

03:18:00 PM 4988488 3062224 38.04 639136 2115404 72404 0.44

03:18:01 PM 4984464 3066248 38.09 639136 2115404 98060 0.60

03:18:02 PM 4985152 3065560 38.08 639136 2115420 97972 0.60

03:18:03 PM 4985400 3065312 38.08 639136 2115420 97972 0.60

[root@localhost] # sar-B 1 100

Linux 2.6.32-220.el6.x86_64 (localhost.localdomain) 07 localhost.localdomain 30 CPU 2013 _ x86 pound 648)

03:19:09 PM pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s vmeff

03:19:10 PM 0.00 157.14 45415.48 0.00 15541.67 0.00 0.00 0.00

03:19:11 PM 0.00 0.00 40.40 0.00 75.76 0.00 0.00 0.00

03:19:12 PM 0.00 0.00 5542.34 0.00 1584.68 0.00 0.00 0.00

[root@localhost] # sar-b 1 100

Linux 2.6.32-220.el6.x86_64 (localhost.localdomain) 07 localhost.localdomain 30 CPU 2013 _ x86 pound 648)

03:19:38 PM tps rtps wtps bread/s bwrtn/s

03:19:39 PM 12.50 0.00 12.50 0.00 181.82

03:19:40 PM 0.00 0.00 0.00

03:19:41 PM 7.14 0.00 7.14 0.00 128.57

[root@localhost] # sar-W 1 100

Linux 2.6.32-220.el6.x86_64 (localhost.localdomain) 07 localhost.localdomain 30 CPU 2013 _ x86 pound 648)

03:20:10 PM pswpin/s pswpout/s

03:20:11 PM 0.00 0.00

03:20:12 PM 0.00 0.00

03:20:13 PM 0.00 0.00

[root@localhost] # sar-d 1 100

Linux 2.6.32-220.el6.x86_64 (localhost.localdomain) 07 localhost.localdomain 30 CPU 2013 _ x86 pound 648)

03:20:48 PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm util

03:20:49 PM dev8-0 7.69 0.00 79.12 10.29 0.00 0.14 0.14 0.11

03:20:49 PM dev8-16 0.00 0.00 0.00

03:20:49 PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm util

03:20:50 PM dev8-0 10.47 0.00 148.84 14.22 0.08 7.22 7.11 7.44

03:20:50 PM dev8-16 0.00 0.00 0.00

7.3 Command output interpretation

Kbmemfree: this value is basically the same as the free value in the free command, so it does not include buffer and cache space.

Kbmemused: this value is basically the same as the used value in the free command, so it includes buffer and cache space.

% memused: this value is a percentage of kbmemused and total memory (excluding swap).

Kbbuffers and kbcached: these two values are buffer and cache in the free command.

Kbcommit: guarantee the memory needed by the current system, that is, the memory needed to ensure that it does not overflow (RAM+swap).

% commit: this value is a percentage of kbcommit to total memory (including swap).

Pgpgin/s: indicates the number of bytes per second from disk or SWAP to memory (KB)

Pgpgout/s: indicates the number of bytes per second from memory to disk or SWAP (KB)

Fault/s: the number of missing pages generated by the system per second, that is, the sum of primary and secondary missing pages (major + minor)

Majflt/s: the number of main missing pages generated per second.

Pgfree/s: the number of pages placed in the idle queue per second

Pgscank/s: number of pages scanned by kswapd per second

Pgscand/s: the number of pages directly scanned per second

Pgsteal/s: the number of pages cleared from cache per second to meet memory requirements

% vmeff: percentage of pages cleared per second (pgsteal) to total scanned pages (pgscank+pgscand)

Tps: the total amount of Istroke O transmissions per second of physical devices

Rtps: the total amount of data read from the physical device per second

Wtps: the total amount of data written to the physical device per second

Bread/s: the amount of data read from a physical device per second, in blocks per second

Bwrtn/s: the amount of data written to the physical device per second, in blocks per second

Pswpin/s: the number of swap pages (swap page) swapped in by the system per second

Pswpout/s: the number of swap pages (swap page) swapped out by the system per second

7.4 Common options

Sar-[u] [r] [B] [b] [Q] [W] [d]

Parameter description:

-u:CPU resource monitoring

-r: memory and swap space monitoring

-B: memory paging monitoring

-b:IO and transmission rate monitoring

-Q: process queue system load monitoring

-W: system exchange partition activity monitoring

-d: equipment usage monitoring

8 tmpfs

8.1 definition

The tmpfs file system is a memory-based file system mounted under / dev/shm.

8.2 Features

Resize dynamically; access is fast and resides completely in the RAM; of course, the system restart is gone.

8.3 usage

1. The POSIX standard shared memory under linux is implemented based on this file system (and a set of System V standard shared memory implementation)

2. Creating files under / dev/shm will directly use tmpfs;, so if you don't need to consider the loss of data after the machine reboot, put the data below to relieve your disk pressure!

3. The default size of tmpfs is half of physical memory. To adjust the maximum capacity to 40G and the number of file nodes to 1000000, you can use the following command:

Mount-o size=40G-o nr_inodes=1000000-o noatime,nodiratime-o remount / dev/shm

4. If you need to permanently modify the size of the tmpfs file system, you need to modify / etc/fstab

9 crontab:

9.1 function

Schedule the execution of some commands on a regular basis, which is equivalent to a system-level timer. There are some files named after the user name in / var/spool/cron/, indicating the scheduled execution of the task by the user.

9.2 configuration method

Format description:

* / command path

The first five fields can be rounded to specify when to start work (minutes, hours, date, month and week), and the sixth field is a string, scripts, programs, etc. that are due to be executed.

43 21 * implemented at 21:43 every day

017 * * 1 every Monday at 17:00

0pr 10 17 * * 0je 2pm 3 executes at 17:00 and 17:10 every Sunday, Tuesday and Wednesday

0-10 17 1 * * from 17:00 to 7:10 on the 1st of January, at an interval of 1 minute

00 1pm 15 * 1 May 1st and 15th and 0:00 on the first day of the first month

42 4 1 * * at 4:42 on the 1st of June

021 * 1-6 from Monday to Saturday at 21:00

0, 10, 10, 20, 30, 40, 50 * every 10 minutes.

* / 10 * every 10 minutes

* 1 * every minute from 1:0 to 1:59

0 1 * 1:00 execution

Execute every hour at 0 * / 1 * 0pm

It is executed every hour at 0 * 0am.

2 8-20 beat 3 * 8 02ju 02ju 02ju 14purl 02ju 17ju 02ju 20purl 02 execute

Execution at 5:30 on the first and 15th of 3051 people

The following configuration indicates that this script / opt/ clear_old_file.sh is executed at 3:00 every day

The function of this script is to delete files that have not been accessed within five days in the two directories / data1 / data2.

0 3 * (cd / opt/ & &. / clear_old_file.sh > / dev/null &)

[root@ShanWei_119_134_255_208 ~] # cat / opt/clear_old_file.sh

#! / bin/sh

Find / data1/*-type f-atime + 5-exec rm {}\

Find / data2/*-type f-atime + 5-exec rm {}\

Exit 0

10 proc: to be continued...

The / proc file system is unique to GNU/Linux. It is a virtual file system that resides entirely in RAM, so no files in that directory consume disk space. Through it, we can easily understand the kernel information and hardware information in the system, and we can also configure the parameters of the system kernel through it. Many commands actually just collect information from / proc files, organize them in their own format and display them; that's basically what the commands described earlier do.

This is the end of the article on "sample analysis of CentOS service program performance evaluation documents". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.