Shulou(Shulou.com)06/01 Report--

This article mainly describes how to solve the memory statistics and memory leakage problems under Linux, the article is very detailed, has a certain reference value, interested friends must read!

Linux in the use of memory on the principle is: if the memory is sufficient, do not use white, try to use memory to cache some files, so as to speed up the process, and when the memory is insufficient, will be through the corresponding memory recycling strategy to recover cache memory, for the process to use.

I. Analysis of total system memory

You can learn the current system memory usage summary from the meminfo file under proc directory, where available physical memory =memfree+buffers+cached. When memfree is not enough, the kernel will write cached and buffered memory back to the backing store through the write-back mechanism (pdflush thread), thus releasing the relevant memory for use by the process, or explicitly releasing cache memory manually:

echo 3 > /proc/sys/vm/drop_caches

The following figure shows the overall usage of the current system memory under the HiSilicon platform. It can be seen that the system consumes 29 MB of memory. The following analysis continues to analyze who consumes these memories.

# cat /proc/meminfoMemTotal: 68956 kB

MemFree: 18632 kB

Buffers: 4096 kB

Cached: 17260 kB

SwapCached: 0 kB

Active: 21304 kB

Inactive: 19248 kB

SwapTotal: 0 kB

SwapFree: 0 kB

Dirty: 0 kB

Writeback: 0 kB

AnonPages: 19216 kB

Mapped: 2472 kB

Slab: 6900 kB

SReclaimable: 924 kB

SUnreclaim: 5976 kB

PageTables: 460 kB

NFS_Unstable: 0 kB

Bounce: 0 kB

CommitLimit: 62060 kB

Committed_AS: 28864 kB

VmallocTotal: 442368 kB

VmallocUsed: 46984 kB

VmallocChunk: 393212 kB

II. Statistics on memory usage by processes

In a 32-bit operating system, each process has 4GB of virtual memory space, of which 0 - 3GB is the private user space of each process, which is invisible to other processes in the system. 3 - 4 GB is Linux kernel space, shared by all processes and kernels of the system. By accessing the file under/proc/{pid}/, you can see how much virtual memory space is being used by each thread, and thus how much memory is being consumed by each thread.

Since our products are implemented using multithreading, multiple threads share the user-state virtual address space of a process. The virtual address space contains several regions, mainly the following regions:

1. The code segment of the current execution file, which is called the text segment.

2. The data segment of the execution file mainly stores global variables and static variables used in the execution file.

Heaps that store global variables and dynamically generated data.

4. Stack for storing local variables and implementing function calls.

5. Memory segments mapped to virtual address space by mmap

So just look at the user-state virtual address space allocation of any thread to know the total memory occupied by all threads belonging to the same process. You can view the/proc/{pid}/maps file to get the relevant virtual address space content. Typical content in the following abstract column:

# cat /proc/568/maps00008000-0036a000 r-xp 00000000 00:0e 236 /home/hik/hicore

00372000-003a5000 rw-p 00362000 00:0e 236 /home/hik/hicore

003a5000-00e28000 rwxp 003a5000 00:00 0 [heap]

40000000-40005000 r-xp 00000000 01:00 94 /lib/ld-uClibc.so.0

416db000-41770000 rw-s c2005000 00:0f 68 /dev/mem

b51fc000-b5200000 rwxp b51fc000 00:00 0

…….

be1fc000-be200000 rwxp be1fc000 00:00 0

be93b000-be950000 rwxp befeb000 00:00 0 [stack]

*** Line: From r-xp, it can be seen that its permission is read-only and executable. The memory address of this section corresponds to the code segment of the execution file. The code segment of the program needs to be loaded into memory before it can be executed. Because it is read-only, it is not modified and is shared throughout the system.

The second line: From rw-p, it can be seen that its permission is readable and writable, but not executable. The memory address of this section corresponds to the data section of the execution file, storing the global variables and static variables used in the execution file.

The third line: from rwxp know that its permissions are readable and writable, executable, address space up growth, and does not correspond to files, is a heap segment, the process uses malloc to apply for memory in the heap segment. Each process has only one heap segment, and memory requested by either the main process or different threads is reflected into the heap segment of the process. The heap segment grows upwards, *** can grow to a position of 1GB, i.e. 0x4000000, if it is larger than 1GB, glibc will use mmap to apply for a block of memory for the heap.

The fourth line: is the memory address of the shared library to which the program is connected.

The fifth line is the virtual address space mapped in mmap mode.

The sixth and seventh lines: are the stack address segments of the thread, and the stack size of each thread is 16K.

Line 8: Stack area of process. Regarding stack segments, there is one for each thread, or multiple stack segments if there are multiple threads in the process.

III. Statistics of total memory of current system

The total memory occupied by the process can be calculated through the maps table above.

2. When the system is running, it will mount the files related to the application layer to the tmpfs file system. This part of the HiSilicon system is about 13 MB. This part of the memory is counted by cache, but this part of the memory cache cannot be released by recycling policy or explicit call.

3. Memory occupied by the root file system ramdisk

4. The size of the memory reserved by the current system can be obtained or modified by viewing/proc/sys/vm/min_free_kbytes.

5, of course, when the system is running, there should also be a certain amount of memory for cache when reading and writing hard disk or allocation of skb when the network load is relatively high, generally more than 30M.

IV. Some enlightenment on debugging memory leak problems

When a process requests memory, it is actually the memory manager built into glibc that receives the request, and as the process requests more memory, the memory manager plunges into the kernel via system calls to allocate more memory to the process.

For heap segment management, the kernel provides two system calls brk and mmap, brk to change the address at the top of the heap and mmap to allocate a virtual address space for a process.

When a process requests memory from glibc, glibc uses mmap to allocate a virtual address space for the process if the amount of memory requested is greater than a threshold, rather than brk to extend the pointer at the top of the heap. By default, this threshold is 128K, and can be modified by the function.

#include

Int mallopt(int param, int value)

Param values are M_MMAP_THRESHOLD and M_MMAP_MAX respectively.

The value of Value is in bytes.

M_MMAP_THRESHOLD is the threshold for requesting chunk memory in glibc. For memory requests above this threshold, the memory manager will request memory using the mmap system call. For memory requests below this threshold, the memory manager will extend the heap top pointer using the brk system call.

M_MMAP_MAX is the maximum number of address segments allocated using mmap in the process.

If in the actual debugging process, suspect that a memory leak occurred somewhere, you can check the maps table of the process to see whether the virtual address space of the heap segment or mmap segment of the process continues to increase, if yes, it is likely that a memory leak occurred, if the virtual address space of the mmap segment continues to increase, you can also see the size of the virtual address space of each segment, so that you can determine how much memory is applied for, which can play a very good role in debugging memory leak problems.

The above is "Linux how to solve the memory statistics and memory leak class problems" all the content of this article, thank you for reading! Hope to share the content to help everyone, more relevant knowledge, welcome to pay attention to the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.