Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the basic knowledge points of Linux memory". The content of the explanation in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what are the basic knowledge points of Linux memory.

1. Background

When it comes to linux memory, we often focus on basic commands such as free,top. When the system encounters an abnormal situation, the root cause of the memory problem can be traced back, and there is a lack of deep-seated debug ability in on-site diagnosis. Without in-depth discussion, this space can clearly describe the problems of the current system, which is the most basic ability that every SRE should have.

2. Free

2.1 principle of free command

Free gets the memory usage by looking at / proc/meminfo. But where did the / proc/meminfo file come from? Let's take a look at the / proc directory:

/ proc is a virtual file system, and all files in this directory are pseudo files. Such files only exist in memory and do not take up space-- you can verify them using du-sh, and the disk occupancy under the mold path is 0.

All files under / proc are virtual entries created by the kernel calling the proc_create () interface.

Most of the files in / proc feed back the real-time information of the system (process, memory, cpu, device information, etc.).

Conclusion: / proc/meminfo is a "pseudo file" that stores your memory-related information under the / proc file system.

2.2 introduction to command output

The output of each release varies somewhat, so let's take the debian8 4.19.x release as an example.

Root@4f996feeb851:~# free-m total used free shared buffers cachedMem: 1991 1909 81 4 155 836 Mozilla + buffers/cache: 917 1073Swap: 1023 1 1022

Most of the command output meaning, you can find parsing in the man document, I will not repeat it here.

Used: memory used used = total-free-buffers-cached

Free: unused memory memFree & swapFree in / proc/meminfo

Shared: memory shmem in / proc/meminfo used by tmpfs

Buffers: memory buffered by the kernel to use

Cached: memory used by page caching and slabs

Buffers/cache: represents the sum of buffers and cache

Swap: usage of swap partitions

2.3 will buffer and cache use memory?

The answer is yes, let's take a look at buffer and cache first.

The official definition of cache (cache) is a space reserved to make up for the mismatch of access speed between high-speed devices and low-speed devices, and to speed up access to resources. To put it simply, you can read faster.

Buffer (buffering) is to do resource write shaping. When a computer encounters a large number of "small-scale IO", it will shape it into a small amount of "large-scale IO" to reduce the number of writes. In order to achieve the effect of rational use of "writing resources".

However, the buffer and cache displayed by the free command have a narrow meaning-the buffer displayed by free represents the cache occupied by the block device, and the cache displayed by free represents the the page cache (cache page) occupied by ordinary files.

In short, the memory used by buffer and cache is used to speed up Linux read and write performance. If a new process needs memory, the system will reclaim the memory occupied by buffer and cache and reallocate it to the process.

2.4 other memory concepts

RSS & VSZ & PSS & USS

RSS (Resident Set Size): the amount of physical memory actually used by the process, including sharedMem.

VSZ (Virtual Memory Size): all the amount of memory that can be accessed by the process, including the amount of memory that has been released by swap due to a page fault, and sharedMem.

PSS (Proportional Set Size): adds the amount of memory to the RSS proportionally.

USS (Unique Set Size): the amount of physical memory exclusive to the process.

UsedMem is divided into active & inactive

Active: indicates that this part of the memory is being used by a particular process and is unlikely to be reclaimed.

Inactive: indicates that this part of the memory is allocated to a process that is not in the running state and may be reclaimed.

Linux maintains a LRU List to manage the collection of active and inactive pages. To put it simply, the closer you get to the end of the List, the more likely the page is to be recycled, while the closer it is to the head of the column, the less likely it is to be recycled. The linux kernel maintains two types of LRUList--active list and inactive list. The newly visited pages are put into active list, the pages that have not been visited for a long time are put into inactive list, and the kernel thread kswapd periodically moves the pages from active list to   inactive list.

If the memory of the system's inactive is too large, you can reclaim it by doing the following.

Sync; echo 3 > / proc/sys/vm/drop_caches

3. Virtual memory

In modern x86 systems, the memory that a computer can use is greater than the upper limit of its physical memory, which depends on the virtual memory mechanism. Linux supports virtual memory mechanism and real mode mechanism.

In real mode, the computer will directly apply for physical memory, and under the virtual memory mechanism, the system will treat the disk as an extension of memory, which has increased the amount of memory that can be used. And through the mechanism of mapping map, the real correspondence between physical memory and physical memory is saved.

The activity of transferring Page between disk and memory is called swapping or page scheduling (paging), and the disk partition used as virtual memory is called swap.

The problem of insufficient memory can be temporarily alleviated by adding swap online, but generally, the operation of reducing swap online cannot be done directly, which is likely to lead to the crash of the process. For more information on how to configure swap, please see 5.1swap related configuration.

4. OOM

1.What is OOM?

Out Of Memory Killer is a system protection mechanism of Linux. When the system memory is tight, kill drops some processes to prevent the system from getting stuck. The system implements the killing operation of the process through the scoring mechanism. The default mechanism is to scan all processes for memory usage, cpu usage, and other factors, and then rate    (badness). The higher the score, the higher the priority of the process by kill.

two。 What behaviors will cause the system to rate the process?

The process uses the fork (2) call, which adds points (+) when many child processes are created.

The process has been running for a long time, or a lot of CPU time has been used, which will reduce the score (-)

If the nice value of the process is low, it will be added (+)

If the process is a privileged process (privileged), the score will be reduced (-)

If the process has direct access to the hardware device, it will lose points (-)

3. Where can I see the score of the process?

/ proc//oom_score

4. Manually adjust the score

/ proc//oom_adj this file can be used to adjust which processes should be kill when oom occurs. The range is-16-- + 15. The default value is 0,   .

Special value-17: indicates that the process will never be kill.

5. How do I know if the system has triggered OOM?

/ var/log/messages, / var/log/syslog Syslog or dmesg Syslog diagnostic tools can be found.

5. Memory related configuration

5.1 swap related configuration

Tell the computer to use the weight of the swap partition by adjusting the system parameters

1. The range of swappiness is 0-100. the default is 600. it means that the use of swap60 is prohibited. The default is 100. it is crazy to use swap2. Operation method # sysctl vm.swappiness=VALUE# sysctl vm.swappiness=20 or # echo VALUE > / proc/sys/vm/swappiness# echo 30 > / proc/sys/vm/swappiness

By increasing the swap partition size online, temporarily controlling memory leaks, insufficient memory and other exceptions.

1. Root user 2. Create a storage file # dd if=/dev/zero of=/home/swap2G bs=1024 count=2M3. Security settings # chown root:root / home/swap2G# chmod 0600 / home/swap2G4. Create the liunx exchange partition # mkswap / home/swap2G5. Enable exchange partition # swapon / home/swap2G6. Update fstab file [Note: some operating systems do not need] # vim / etc/fstab/home/swap2G none swap sw 007. Check to see if it works # free-M8. Uninstall swap partition # swapoff / home/swap2G

5.2 cache dependent

Sync; echo 3 > / proc/sys/vm/drop_caches0: do not release 1: release page cache 2: release dentries and inodes3: release all caches

5.3 OOM related

Vm.panic_on_oom

Whether kernel panic is triggered when the oom mechanism is triggered. 0 means off (recommended) and 1 means on. Kernel panic is an action that occurs when the computer encounters a fatal error and doesn't know what to do with it-an analogy to the blue screen of windows. We certainly don't want to have a blue screen every time the computer is in oom. Recommended setting to 0

Vm.overcommit_kbytes:

It is used to limit the maximum memory that a process can request. 0 means it is not set. If you set another value, such as 400, the maximum memory that a process can apply for is swap+400kBytes.

Vm.overcommit_ratio:

Defines the maximum memory that the process can use (percentage mode), which defaults to 50. Indicates that after configuring 50, the process is not allowed to request memory that exceeds swap + 50% * total physical memory.

Vm.oom_kill_allocating_task (Linux 2.6.24 + support)

This enables or disables the kill OOM trigger task when there is insufficient memory. 0 means disabled (default) and 1 means enabled. It can be understood as the switch of the oom mechanism, which is disabled by default-- which means that you want the oom trigger to execute properly.

If you are interested, you can man proc yourself.

Thank you for your reading, these are the contents of "what are the basic knowledge points of Linux memory?" after the study of this article, I believe you have a deeper understanding of what the basic knowledge points of Linux memory have, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.