Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to learn Linux memory management from the developer's point of view. Many people may not know much about it. In order to let everyone know more, Xiaobian summarizes the following contents for everyone. I hope everyone can gain something according to this article.

Memory management has always been the focus of all operating system books, whether on the market or online are full of a large number of teaching materials and information related to memory management. Therefore, we are here to write Linux memory management to take the strategy of avoiding the important, from the theoretical level will not go to teach an axe, a joke. What we want to do and what we can do is talk about memory management from a developer perspective, with the ultimate goal of sharing our experience with memory in kernel development and our understanding of Linux memory management.

How does the process use memory?

There is no doubt that all processes (executing programs) must occupy a certain amount of memory, either for program code loaded from disk, for data taken from user input, and so on. However, the way processes manage this memory varies according to the use of memory. Some memory is statically allocated and uniformly reclaimed in advance, while others are dynamically allocated and reclaimed as needed.

For any normal process, it involves five different data segments. Friends with a little programming knowledge can think of these data segments containing "program code segments,""program data segments,""program stack segments," etc. Yes, all of these data segments are included, but in addition to the above data segments, the process contains two additional data segments. Let's briefly summarize the five different data areas contained in the memory space corresponding to the process.

Code segment: A code segment is an instruction that stores an executable file, that is, it is a mirror image of the executable program in memory. Code snippets need to be protected from illegal modification at runtime, so only reads are allowed, not writes (modifications)--it is not writable.

Data segments: Data segments are used to store initialized global variables in an executable file, in other words, statically allocated variables and global variables.

BSS segment: The BSS segment contains global variables that are not initialized in the program, and the bss segment is set to zero in memory.

Heap: Heap is used to store dynamically allocated memory segments during process operation. Its size is not fixed and can be dynamically expanded or reduced. When a process allocates memory by calling a function such as malloc, the newly allocated memory is dynamically added to the heap (heap is expanded); when a function such as free is used to free memory, the freed memory is removed from the heap (heap is reduced).

Stack: A stack is a temporary local variable created by the user repository, that is, a variable defined in our function parenthesis "{}"(but excluding statically declared variables, static means storing variables in data segments). In addition, when a function is called, its parameters are pushed onto the stack of the calling process, and at the end of the call, the return value of the function is placed back on the stack. Stacks are particularly convenient for saving/restoring call sites due to their first-in, first-out nature. In this sense, we can think of the stack as a memory area where temporary data is stored and exchanged.

How does the process organize these regions?

Data segments, BSS, and heaps in these memory areas are usually stored contiguously-memory locations are contiguous, while code segments and stacks tend to be stored independently. Interestingly, the relationship between heap and stack is very "ambiguous". They are "long" downward (stack down and stack up in i386 architecture) and "long" upward. But you don't have to worry about them meeting, because they are so far apart (how far apart, you can calculate from the example program below) that they rarely meet.

The following figure briefly describes the distribution of the process memory regions:

"Facts speak louder than words," we use a small example (from User-Level Memory Management) to show the differences and locations of the various memory areas mentioned above.

#include

The result is as follows

below are addresses of types of process's memText location:Address of main(Code Segment):0x8048388____________________________Stack Location:Initial end of stack:0xbffffab4new end of stack:0xbffffab0____________________________Data Location:Address of data_var(Data Segment):0x8049758New end of data_var(Data Segment):0x804975c__________________________BSS Location:Address of bss_var:0x8049864____________________________Heap Location:Initial end of heap:0x8049868New end of heap:0x804986c

You can also see the size of each segment of the program using the size command. For example, executing the size example will get

text data bss dec hex filename1654 280 8 1942 796 example

But these data are static statistics of program compilation, while the dynamic values shown above are process runtime, but the two are corresponding.

With the previous example, we have a sneak peek at the logical memory distribution used by processes. In this section, we will continue to enter the operating system kernel to see how the process allocates and manages memory.

From the user's perspective to the kernel, the memory representation used goes through logical addresses, linear addresses, and physical addresses (the explanation of several addresses has been described earlier). Logical addresses are transformed into linear addresses by segment mechanism, and linear addresses are transformed into physical addresses by page mechanism. (However, we should know that although the Linux system retains the segment mechanism, it sets the segment address of all programs to 0-4G, so although logical address and linear address are two different address spaces, in Linux logical address is equal to linear address, and their values are the same). Following this thread, the main problems we studied focused on the following.

1. How to manage process space addresses?

2. How do process addresses map to physical memory?

3. How is physical memory managed?

And some of the sub-problems that arise from the above. Such as system virtual address distribution; memory allocation interface; continuous memory allocation and discontinuous memory allocation.

process memory space

Linux operating system uses virtual memory management technology, so that each process has its own process address space. This space is a linear virtual space with a block size of 4G. What the user sees and touches is the virtual address, and the actual physical memory address cannot be seen. Using virtual addresses not only protects the operating system (users cannot access physical memory directly), but more importantly, user programs can use a larger address space than physical memory (see Hardware Basics for specific reasons).

Before discussing the details of the process space, here are a few clarifications:

*** The process address space of 4G is artificially divided into two parts-user space and kernel space. User space runs from 0 to 3G (0xC000000) and kernel space occupies 3G to 4G. User processes can only access user-space virtual addresses, not kernel-space virtual addresses. Kernel space can only be accessed at times such as when a user process makes a system call (representing a user process executing in kernel mode).

Second, the user space corresponds to the process, so whenever the process switches, the user space will change; while the kernel space is responsible for mapping by the kernel, it will not change with the process, it is fixed. Kernel-space addresses have their own page tables (init_mm.pgd), and user processes have their own page tables.

Third, the user space of each process is completely independent and unrelated. Believe it or not, you can run the above program 10 times simultaneously (of course, to run simultaneously, let them sleep together for 100 seconds before returning), and you will see that 10 processes occupy exactly the same linear address.

Process memory management

The objects of process memory management are memory mirrors on the linear address space of the process, which are actually virtual memory regions used by the process. A process virtual space is a 32-or 64-bit "flat"(independent contiguous interval) address space (the exact size of the space depends on the architecture). To manage such a large flat space uniformly is no easy task. For ease of management, virtual space is divided into many memory areas of variable size (but must be multiples of 4096), which are arranged in the linear address of the process like parking spaces. The division principle of these areas is "store the address space with consistent access attributes together," and the so-called access attributes here simply refer to "readable, writable, executable, etc."

If you want to see the memory area occupied by a process, you can use the command cat /proc//maps (pid is the process number, you can run the example we gave above--./ example &;pid will print to the screen), you can find a lot of digital information similar to the following.

Because the program example uses a dynamic library, it contains memory areas used by the dynamic library in addition to the memory areas used by the example itself (in the order of regions: code segment, data segment, bss segment).

We only extract the information related to example below. In addition to the code segment and data segment represented by the first two lines, the *** line is the stack space used by the process.

08048000-08049000 r-xp 00000000 03:03 439029 /home/mm/src/example 08049000-0804a000 rw-p 00000000 03:03 439029 /home/mm/src/examplebfffe000 - c0000000 rwxp ffff000 00:00 0

Each row of data is formatted as follows:

(memory area) start-end access rights offset major device number: minor device number i node file.

Note that you must have noticed that the process space contains only three memory regions, and there seems to be no heap, bss, etc. mentioned above. In fact, this is not the case. The program memory segment and the memory region in the process address space are fuzzy mappings, that is, the heap, bss, and data segments (initialized) are all represented by the data segment memory region in the process space.

In Linux, the data structure corresponding to the process memory area is vm_area_struct. The kernel manages each memory area as a separate memory object, and the corresponding operations are consistent. The object-oriented approach allows VMA structures to represent multiple types of memory regions--such as memory-mapped files or user-space stacks for processes--and to operate on them differently.

vm_area_strcut structure is more complex, for its detailed structure, please refer to the relevant information. We will only make a few additional remarks about its organization. vm_area_struct is the basic management unit describing the process address space. For a process, it often needs multiple memory areas to describe its virtual space. How to associate these different memory areas? You might think of lists, and indeed vm_area_struct structures are linked in lists, but for ease of lookup, the kernel organizes memory areas in red-black trees (previous kernels used balanced trees) to reduce search time. Two forms of organization coexist, not redundant: linked lists are used when all nodes need to be traversed, and red-black trees are used when specific memory regions are located in the address space. The kernel uses both data structures in order to achieve high performance for various operations on memory areas.

The following diagram reflects the management model of the process address space:

The description structure corresponding to the address space of a process is the "memory descriptor structure", which represents the entire address space of the process, --containing all information related to the address space of the process, including, of course, the memory area of the process.

Having read all of this, do you have any idea how to learn Linux memory management from a developer's perspective? If you still want to know more knowledge or related content, please pay attention to the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.