Shulou(Shulou.com)06/02 Report--

Relative to SMP (also known as uniform. Memory architecture), NUMA introduced the concept of node, each physical CPU is treated as a node, and each node has a local memory (faster access), and the memory of other node is remote

NUMA-related policy 1, each process (or thread) inherits the NUMA policy from the parent process and assigns a priority node. If the NUMA policy allows, the process can call resources on other node. 2. The CPU allocation strategies of NUMA include cpunodebind and physcpubind. Cpunodebind specifies that the process runs on several node, while physcpubind can specify more precisely which cores to run on. 3. The memory allocation strategies of NUMA include localalloc, preferred, membind and interleave. Localalloc specifies that the process requests memory allocation from the current node; preferred loosely specifies a recommended node to acquire memory, and if there is not enough memory on the recommended node, the process can try another node. Membind can specify several node, and the process can only request memory allocation from these specified node. Interleave specifies that the process interweaves requests for memory allocation with the Round-roll algorithm from a specified number of node.

In existing Redhat Linux, localalloc is the default NUMA memory allocation policy, and this configuration option makes it easy for a resource monopolizer to run out of memory for a node. When a node runs out of memory, Linux allocates the node to a process (or thread) that consumes a lot of memory, and it doesn't help even if other node has free memory, so swap is generated.

You can view related information through numactl numactl-- hardware View # numactl-- hardwareavailable: 2 nodes (0-1) node 0 size: 32276 MBnode 0 free: 26856 MBnode 1 size: 32320 MBnode 1 free: 26897 MBnode distances:node 0 10: 10 21 1: 21 10

The numa information assigned to mysqld can be viewed through / proc/pid/numa-maps-reading numa-maps will block the corresponding process, so it is not appropriate to call 2aaaaad3e000 default anon=13240527 dirty=13223315 swapcache=3440324 active=13202235 N0calls 7865429 N1 5375098 frequently.

Output can be formatted through perl script N0: 7983584 (30.45 GB) N1: 5440464 (20.75 GB) active: 13406601 (51.14 GB) anon: 13422697 (51.20 GB) dirty: 13407242 (51.14 GB) mapmax: 977 (0.00 GB) mapped: 1377 (GB) swapcache: 3619780 (13.81 GB)

Node 0x1 allocates 32 gigabytes each, but node 0 has used more than 30 gigabytes and there is little left. If you continue to request memory at node 0, you need swap, even if node 1 still has 10 gigabytes left. Solution: use interleave to poll each node for memory allocation modification mysqld_safe, and add the following information after cmd= "$NOHUP_NICENESS": cmd= "/ usr/bin/numactl-- interleave all $cmd" when NUMA memory allocation N0: 6814756 (26.00 GB) N1: 6816444 (26.00 GB) anon: 13629853 (51.99 GB) dirty: 13629853 (51.99 GB) mapmax: 296 (0.00 GB) mapped: 1384 (GB)

Twitter production library 144G of memory, innodb buffer pool occupies 120g; in addition to using numactl-- interleave=all call sysctl-Q-w vm.drop_caches=3 before starting mysqld to clear the cache and start mysqld to allocate innodb buffer pool immediately

Note: NUMA can also be changed to SUMA at the BIOS level, and all allocation policies can be set to interleave at the bottom, but too large granularity is not conducive to differentiation. The default innodb buffer pool uses lazy-allocation, which can be allocated immediately by-- innodb-buffer-pool-populate, and can be realized by using the MAP_POPULATE bit of mmap.

Reference material http://blog.wl0.org/2012/09/checking-procnuma_maps-can-be-dangerous-for-mysql-client-connections/ http://blog.jcole.us/2010/09/28/mysql-swap-insanity-and-the-numa-architecture/ http://www.realzyy.com/?p=1245#more-1245 http://kevinclosson.wordpress.com/2009/05/14/you-buy-a-numa-system-oracle-says-disable-numa-what-gives-part-ii/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.