Shulou(Shulou.com)06/01 Report--

This article shares with you the content of a sample analysis of virtual memory in linux. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

What is virtual storage? Why do you need it?

We know that program code and data must reside in memory in order to run, but the amount of memory in the system is very limited, often unable to hold all the code and data of a complete program, not to mention in a multitasking system, it may be necessary to open subprocessors, drawing programs, browsers and many other tasks at the same time, so it is obviously impossible for memory to host all of these programs. So the first thing you can think of is to divide the program into small parts, leaving only all the parts needed by the current system to run it in memory and the rest on the hard disk. When the system has finished processing the current task segment, it will call in the next task segment to be run from out-of-memory. Indeed, this is how old systems handle big tasks, and the work is done by programmers themselves. However, as the program language becomes more and more advanced, programmers become less dependent on the system architecture, and few programmers can control the system very clearly, so let the programmer be responsible for fragmenting the program and calling it in on demand will reduce efficiency, and heavy will cause the machine to crash. Another reason is that as programs become richer and richer, it is almost impossible to predict their behavior accurately, making it difficult for programmers to determine which program to load next. Therefore, it is difficult to rely on predictability to statically allocate a fixed size of memory, and then mechanically rotate the program chip into memory for execution. The system must adopt a new technology that can be distributed on demand without programmer intervention.

Virtual memory (called virtual memory) is relative to the logical memory and physical memory in the system. Logical memory is the memory seen from the point of view of the process, so it is the programmer's concern. The physical memory is the memory seen from the processor's point of view, which is managed by the operating system. Virtual memory can be said to be a technical means of mapping to memory from these two different perspectives. ) technology is a method of on-demand dynamic memory allocation taken over by the operating system, it allows programs to unwittingly use storage space larger than the actual physical space (in fact, the storage space needed by the program is scattered in physical memory and disk in the form of pages), so virtual memory completely liberates programmers, so programmers do not have to care too much about program size and loading Free to write programs, tedious things are left to the operating system to do.

Implement virtual memory

Virtual memory combines the hard disk space of the system and the actual memory of the system for the process to use, providing the process with a much larger virtual space than memory. When the program is running, only a small part of the virtual address space is mapped to memory, and the rest is stored on the hard disk (that is, the program virtual space is equal to the actual physical memory plus part of the hard disk space). When the accessed virtual address is not in memory, it means that the address is not mapped to memory, but is stored in the hard disk, so the required virtual storage address is immediately transferred to memory; at the same time, when the system memory is tight, the currently unused virtual storage space can also be swapped out to the hard disk to free up physical memory space. The system works over and over again-in and out, which is almost imperceptible to the user, thanks to the virtual memory mechanism.

The swap partition of Linux is the space reserved by the hard disk for virtual storage space. The experience should be about twice the size of memory. If you are interested, you can use swapon-s to view the swap partition size.

It is easy to understand that it is nothing more than using memory and hard disk space to form virtual memory space. But the address mapping (virtual address mapping to physical address mapping) and virtual address swapping and swapping out repeatedly in this process deserve careful consideration. How exactly does the system map virtual addresses to physical addresses? How can the memory be constantly swapped in and out of the virtual address between the hard disk and memory?

Can the above questions be answered by using the paragraph mechanism? After passing the segment mechanism, the logical address becomes a 32-bit address, which is enough to cover 4G memory space. When the virtual address needed by the program is not in memory, it is difficult to swap the virtual space in and out only by relying on the segment mechanism. Because it is not convenient to transfer the entire size of virtual space between memory and hard disk (in the old system, the whole memory or even the whole process will be clumsily swapped out. Think about the consequences of doing so. ). Therefore, it is necessary to find a smaller and more flexible storage representation unit, so that it is convenient for virtual addresses to call in and out between the hard disk and memory. This smaller storage management unit is the page (4K size). The mechanism that manages page swapping is called the page mechanism.

Because of the use of page mechanism, the address converted by segment mechanism is only as an intermediate address-linear address, which does not represent the actual physical address, but represents the virtual space address of the whole process. On the basis of linear address, the page mechanism then deals with linear address mapping: when the required linear address (virtual space address) is not in memory, the required virtual memory is transferred from disk in pages; when there is not enough memory, the virtual space in memory is swapped out to disk in pages. It can be seen that using pages to manage memory and disk (virtual memory) greatly facilitates the work of memory management. There is no doubt that the page mechanism and virtual memory management are a perfect match.

Using the page mechanism, 4G space is divided into 2 to the 20th power of 4K pages (pages can also be set to 4m size), so each index entry in the index table (page table) needed to locate the page needs at least 20 bits. However, in the page table items often need to add some page properties, so the page table entries are actually 32 bits, of which 12 bits are used to store information such as "whether the page exists in memory" or "page permissions".

We mentioned earlier that the linear address is 32 bits. The top 20 bits index the page table, while the lower 12 bits give the offset in the page. The linear address goes through the page table to find the page base address and the low 12-bit offset is added to form the final physical address.

In practical use, not all page table items are stored in a large page table, because each page table item occupies 4 bytes, if you want to store 2 to the 20th power of page table items in a table, you need 4m continuous storage space. Such a large contiguous space is not easy to find, so the page table is often stored hierarchically, such as two levels, so each level of page table only needs 4k contiguous space.

Two-level page table search is like reading a chapter novel, first find out which chapter it is in, and then find which section under that chapter. Take a look at the following figure:

To sum up, address translation requires two technologies, one is segment mechanism, the other is page mechanism. The segment mechanism deals with the mapping of logical addresses to linear addresses, while the page mechanism is responsible for mapping linear addresses to physical addresses. Together, the two-level mapping accomplishes the daunting task of translating from the logical address seen by the programmer to the physical address seen by the processor.

You can compare these two mechanisms to an address translation function, the variable of the segment mechanism is the logical address, and the function value is the linear address; the variable of the page mechanism is the linear address, and the function value is the physical address. The address translation process is shown below.

Logical address-(segment function)-> linear address-(page function)-> physical address.

Although both segment mechanism and page mechanism participate in mapping, they have different division of labor, and they are independent and do not interfere with each other, so they do not need to know whether each other exists or not.

Let's take a brief look at how the segmented page mechanism is used with an example of Linux.

Segmentation Strategy in Linux

The segment mechanism is limited in Linux and has not been fully utilized. Each task does not arrange its own independent data segment and code segment, but only minimally uses the segment mechanism to isolate user data and system data-Linux only arranges four segments with the same range, kernel data segment, kernel code segment, user data segment, user code segment, they all cover 0-4G space, except that the attributes of each segment are different, kernel segment privilege level is 0, user segment privilege level is 3. This segmentation avoids the translation step from logical address to linear address (logical address is equal to linear address), but still retains the most basic protection of segment level.

Each user process can see the linear space of 4G, in which 0-3G is the user space, which can be directly accessed by the user-mode process; from the 3G-4G space as the kernel space, the kernel code and data are stored, and only the kernel-mode process can be accessed directly, and the user-mode process can not directly access it, so it can only enter the kernel space through system calls and interrupts.

When it comes to privilege switching, we can't do without the concepts of task door, trap door / interruption door and so on. Trap gates and interrupt gates are channels into kernel space when traps and interrupts occur. The call gate is the channel that user space programs need to access each other, the task gate is special, it does not contain any address, but serves for task switching (but linux task switching does not really use it, it is too troublesome).

For all kinds of door systems, there will be corresponding door descriptors, similar to the structure of segment descriptors, door descriptors are also indexed by the corresponding door selection words, and eventually generate a pointer to the offset address within a specific segment. This pointer points to the entrance to be entered. The purpose of using the gate is to ensure that the entrance is controllable so that it does not enter a location in the kernel that should not be accessed.

Paging Strategy in Linux

See how paging is used in linux.

Each process in Linux has its own page table, that is, the mapping function of each process is different, which ensures that the virtual address of each process will not be mapped to the same physical address. This is because processes must be independent of each other and their respective data must be isolated to prevent information disclosure.

It should be noted that the kernel, as a separate part that must be protected, has its own independent page table to map kernel space (not all space, just physical memory size), and this page table (swapper_pg_dir) is statically allocated, it only maps kernel space (swapper_pg_dir only uses items after 768 items-768 page directories can map 3G space). This independent page table ensures that the kernel virtual space is independent of other user program spaces, that is, other processes are normally not related to the kernel (when compiling the kernel, the kernel code is designated to link to more than 3G space). As a result, kernel data is naturally protected.

So what do you do when a user process needs to access kernel space?

Linux takes an ingenious approach: the first 768 items of the user's process page table map the process space (

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.