Shulou(Shulou.com)05/31 Report--

This article mainly explains "the usage of Swap under Linux". The explanation in this article is simple and clear, easy to learn and understand. Please follow the ideas of Xiaobian and go deep into it slowly to study and learn "the usage of Swap under Linux" together!

Linux operating system performance analysis mainly includes disk IO, CPU, memory and network traffic, and here mainly for the system memory usage process to do an analysis.

How to view system memory usage

1, according to common commands to view the system memory usage overview

free -g

total used free shared buffers cached

Mem: 31 31 0 0 0 3

-/+ buffers/cache: 28 3

Swap: 15 7 7

(According to the free command, you can see that the system uses 28G of physical memory and 3 G of remaining memory, of which swap has a total of 15G and 7G has been used)

top （f p M）

top - 14:18:50 up 1280 days, 7:15, 1 user, load average: 1.05, 1.25, 1.12

Tasks: 229 total, 1 running, 227 sleeping, 0 stopped, 1 zombie

Cpu(s): 0.2%us, 0.1%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 32949816k total, 32848840k used, 100976k free, 169308k buffers

Swap: 16771776k total, 8384616k used, 8387160k free, 3276360k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP DATA COMMAND

12428 mysql 15 0 22.2g 20g 3964 S 3.3 66.5 91946:06 1.3g 22g mysqld

22840 cyldj 15 0 9059m 6.7g 9012 S 0.0 21.5 22:05.42 2.1g 8.7g DBAgent

28689 root 15 0 358m 30m 3036 S 0.0 0.1 187:37.41 328m 137m salt-minion

30768 cyldj 21 0 462m 10m 1908 S 0.0 0.0 10:44.60 451m 373m java

22567 root 15 0 86004 3292 2576 S 0.0 0.0 0:00.01 80m 688 sshd

28690 root 20 0 267m 2188 704 S 0.0 0.0 0:00.00 264m 47m salt-minion

661 root 18 0 16340 1836 1632 S 0.0 0.0 0:47.42 14m 308 zabbix_agentd

22569 root 15 0 68156 1520 1188 S 0.0 0.0 0:00.02 65m 408 bash

2901 root 18 0 197m 1336 884 S 0.0 0.0 4:04.57 196m 174m AxisAgent

11236 root 15 0 60672 1324 760 S 0.0 0.0 2:01.21 57m 608 sshd

665 root 15 0 18432 1260 992 S 0.0 0.0 11:39.82 16m 308 zabbix_agentd

662 root 15 0 18432 1256 992 S 0.0 0.0 11:43.36 16m 308 zabbix_agentd

(According to the top command, you can see that the process that uses the most memory is mysql process, followed by dbagent, where dbagent uses more swap partitions)

nmon (m)

Nmon download address:

https://sourceforge.net/projects/nmon/files/nmon_x86_64_rhel5/download （RHEL5）

https://sourceforge.net/projects/nmon/files/nmon_linux_14i_newer_Linux_versions.tar.gz/download （RHEL6）

(You can see the detailed allocation of memory through nmon, where the total memory is 32177.6 MB, the total swap 16378.7 MB , the free 93.4 MB, the recoverable 3188.2+166.5 MB, the used swap partition 7273.6 MB, the used active data size is 30099.9 MB, and the data size that can be swapped out of memory is 1590.7 MB)

2. Check system memory usage according to system monitoring

(memory usage in the last 7 days)

3. View system memory usage information for specified processes

cat /proc/22840/statm

#virt res shr text lib data dt

2319113 1768037 2253 3836 0 2292997 0

(The above is to view the memory usage information of dbagent, where the unit is the number of pages, so it depends on the system page size of 4K, the first column is virtual memory 2319113 pages, the second column is resident memory 1768037 pages, the third column is shared memory 2253 pages, and the process exclusive memory size is res-shr)

getconf PAGESIZE

4096

awk '/^Swap:/ {SWAP+=$2}END{print SWAP" KB"}' /proc/${pid}/smap

for i in `cd /proc;ls |grep "^[0-9]"|awk ' $0 >100'` ;do awk '/Swap:/{a=a+$2}END{print '"$i"',a/1024"M"}' /proc/$i/smaps ;done |sort -k2nr

(The above command can check the virtual memory usage of the process)

cat /proc/22840/status |grep Vm

VmPeak: 9341632 kB

VmSize: 9276452 kB

VmLck: 0 kB

VmHWM: 7074296 kB

VmRSS: 7073540 kB

VmData: 9171904 kB

VmStk: 84 kB

VmExe: 15344 kB

VmLib: 4728 kB

VmPTE: 15596 kB

(The above command can view the memory usage of the specified process. VmSize represents the virtual memory size, and VmRSS represents the resident memory size.)

Second, comprehensive analysis of MYSQL database memory usage

sh show_mem_usage.sh

Global Memory Size: 18592.00 MB

Single thread maximum memory:25.18 MB

Maximum thread memory consumption: 8815.62 MB

Historical maximum thread memory consumption: 5213.81 MB

Innodb Buffer Pool Usage: 99.00%

From the memory usage of mysql and the total physical memory usage, the memory usage of the server has basically reached the upper limit (mysql resident memory is about 21G, dbagent is about 7G, the total physical memory is 31G, the cache is 3 G, and there is about 2 G of inactive memory). When the mysql process and dbagent use more than 5 G of memory, the physical memory is insufficient, and the swap partition will be used.

3. Under what circumstances will swap partition be used

1. If physical memory is really insufficient, swap partition will be used.

2. There is still enough memory in physical memory, such as swap partition used when cache,buffer has more remaining memory.

IV. Use swap partition extension

If there is enough physical memory, swap partition is still used. You can see why swap partition is used from the following aspects

1. Swap partition proportion adjustment

cat /etc/sysctl.conf |grep swap

vm.swappiness=0

A setting of 0 prioritizes maximum use of physical memory for caching data rather than disks as partitions, and higher values are less conducive to full utilization of physical memory

2. Numa trap

In order to improve the cpu and memory data access speed, concurrency, the design of numa architecture, but if numa memory allocation strategy is unreasonable, then it will seriously affect the use of memory, especially for large memory block use of database servers.

#numactl --hardware

available: 2 nodes (0-1)

node 0 size: 16160 MB

node 0 free: 15 MB

node 1 size: 16131 MB

node 1 free: 78 MB

node distances:

node 0 1

0: 10 20

1: 20 10

(The above command can check the physical memory allocation of each node. You can see that the memory allocation ratio of the two nodes is basically equal. If the free values of node0 and node1 are significantly different, it means that there is a problem with the allocation policy, which is likely to cause swap use and physical memory idle state)

You can also confirm this by looking at numa stat to see numa's miss and hit ratio

#numastat

node0 node1

numa_hit 39150779957 38736256884

numa_miss 2658848763 8851827358

numa_foreign 8851827296 2658848763

interleave_hit 122652306 137287417

local_node 39096884744 38598664497

other_node 2712743976 8989419745

3. Solution

(1) Enable large page management

(2) Close numa

***BIOS OFF NUMA

*** Start kernel shutdown:

vi /boot/grub/grub.conf

kernel /boot/vmlinuz-2.6.18-128.1.16.0.1.el5 root=LABEL=DBSYS ro bootarea=dbsys rhgb quiet console=ttyS0,115200n8 console=tty1 crashkernel=128M@16M numa=off

Restart the operating system and check whether it is closed by cat /proc/cmdline and numactl --hardware

(3) Adjust kernel parameters

Kernel parameter: zone_reclaim_mode, if 0, then the system will tend to allocate memory from other nodes, if 1 means that the system will tend to reclaim Cache memory from local nodes most of the time

vm.swappiness=0

echo 0 > /proc/sys/vm/zone_reclaim_mode ; echo "vm.zone_reclaim_mode = 0" >> /etc/sysctl.conf

(4) Adjust kernel parameters

Mysql can adjust relevant parameters and cross-start methods to improve numa, such as numactl --interleave=all /etc/init.d/mysql start

localloc specifies that the process requests memory allocation from the current node;

preferred loosely specifies a recommended node to acquire memory, and if there is not enough memory on the recommended node, the process can try another node.

membind can specify several nodes from which a process can only request memory allocation.

Interleave specifies that a process requests memory allocation from a specified number of nodes interleaved using RR (Round Robin Scheduling) algorithm.

#NUMA support

numa_interleave = 1innodb_buffer_pool_populate = 1flush_caches=1 Thank you for reading, the above is the content of "Swap usage under Linux", after learning this article, I believe that everyone has a deeper understanding of Swap usage under Linux, and the specific use needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.