Shulou(Shulou.com)06/01 Report--

Most people do not understand the knowledge points of this article "what is Linux virtual memory?", so the editor summarizes the following content, detailed content, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this "what is Linux virtual memory" article.

The origin of virtual memory

There is no doubt that virtual memory is one of the most important concepts in the operating system. I think it is mainly due to the important "strategic position" of memory. CPU is too fast, but its capacity is small and its function is single. Other Imax O hardware supports a variety of fancy functions, but they are too slow compared to CPU. So there is a need for a lubricant between them as a buffer, which is where memory shows off.

In modern operating systems, multitasking is standard. Multi-task parallelism greatly improves the utilization of CPU, but it leads to the conflict of memory operations among multiple processes. The concept of virtual memory is proposed to solve this problem.

The figure above is the simplest and most intuitive explanation of virtual memory.

The operating system has a piece of physical memory (the middle part) and two processes (actually more) P1 and P2. The operating system secretly told P1 and P2 that my entire memory is yours. But in fact, the operating system just gives them a big pie. The memory is said to be given to P1 and P2, but in fact it only gives them a serial number. Only when P1 and P2 really began to use this memory, the system began to use toss and move, piecing together blocks for the process. P2 thought it was using A memory, but in fact it had been quietly redirected to the real B by the system, even when P1 and P2 shared C memory, they didn't know.

This means of deceiving the process of the operating system is virtual memory. For processes such as P1 and P2, they all think they are occupying the entire memory, and they don't know or care about which address of the physical memory they use.

Paging and page table

Virtual memory is a concept in the operating system, for the operating system, virtual memory is a comparison table. P1 should go to the An address of the physical memory when getting the data in A memory, and the data in B memory should go to the C address of the physical memory.

We know that the basic units in the system are Byte bytes, if each Byte of virtual memory corresponds to the address of physical memory, and each entry needs at least 8 bytes (32-bit virtual address-> 32-bit physical address), in the case of 4G memory, you need the space of 32GB to store the comparison table, then this table is too large to fit the real physical address, so the operating system introduced the concept of Page.

When the system boots, the operating system divides the entire physical memory into pages in 4K units. After the memory allocation, are in pages, then the virtual memory page corresponding to the physical memory page mapping table will be greatly reduced, 4G memory, only need 8m mapping table, some processes do not use the virtual memory, and do not need to save the mapping relationship, and Linux also designed a multi-level page table for large memory, can enter one page to reduce memory consumption. The mapping table of operating system virtual memory to physical memory is called a page table.

Memory addressing and allocation

We know that through the virtual memory mechanism, each process thinks that it occupies all the memory. When the process accesses the memory, the operating system will convert the virtual memory address provided by the process into a physical address, and then go to the corresponding physical address to obtain data. There is a kind of hardware in CPU, and the memory management unit MMU (Memory Management Unit) is specifically used to translate virtual memory addresses. CPU also sets a caching policy for page table addressing, and its cache hit rate can reach 98% because of the locality of the program.

The above situation is that there is a mapping from virtual address to physical address in the page table, and if the physical address accessed by the process has not been assigned, the system will generate a page fault interrupt. The system cuts to the kernel state to assign the physical address to the process virtual address.

Function

Virtual memory not only solves the problem of memory conflicts accessed by multiple processes through memory address translation, but also brings more benefits.

Process memory management

It helps processes manage memory, mainly in the following ways:

Memory integrity: due to the "spoofing" of virtual memory to processes, each process thinks that the memory it acquires is a contiguous address. When we write an application, we don't have to think about the allocation of chunks of addresses, and we always think that the system has enough chunks of memory.

Security: because the process accesses memory, it is addressed through the page table, and the operating system adds a variety of access permission identification bits to each item of the page table to realize the access control of memory.

Data sharing

It is easier to share memory and data through virtual memory.

When the process loads the system library, it always allocates a piece of memory to load the library files from the disk into this memory. When the physical memory is used directly, because the physical memory address is unique, even if the system finds that the same library is loaded twice in the system, the system is powerless because the loading memory specified by each process is different.

When using virtual memory, the system only needs to point the virtual memory address of the process to the physical memory address where the library file is located. As shown in the figure above, the B addresses of processes P1 and P2 point to the physical address C.

It is also easy to use shared memory by using virtual memory. The system only needs to point the virtual memory address of each process to the shared memory address assigned by the system.

SWAP

Virtual memory allows processes to "expand" memory.

We mentioned earlier that virtual memory allocates physical memory to processes through page fault interrupts. Memory is always limited. What if all physical memory is occupied?

Linux puts forward the concept of SWAP. SWAP partition can be used in Linux. When the physical memory is allocated, but the available memory is insufficient, the temporarily unused memory data is put on disk first, so that the processes in need use the data first, and then the data is loaded into memory when the process needs to use the data again. Through this "swap" technology, Linux allows the process to use more memory.

common problem

I also have a lot of problems when learning about virtual memory.

32 bit and 64 bit 3

The most common are 32-bit and 64-bit problems.

CPU accesses memory through physical buses, so the range of access addresses is limited by the number of machine buses. On 32-bit machines, there are 32 buses, and each bus has high and low potentials representing 1 and 0 of bit, respectively, so the maximum accessible address is 2 ^ 32bit = 4GB, so it is invalid to insert memory greater than 4G on 32-bit machines, and CPU can access less than 4G of memory.

However, 64-bit machines do not have a 64-bit bus, and their maximum memory is limited to the operating system. Linux currently supports up to 256GB of memory.

According to the concept of virtual memory, it is not impossible to run 64-bit software on a 32-bit system, but because of the structural design of the virtual memory address, the 64-bit virtual address can not be used in the 32-bit system.

Direct manipulation of physical memory

The operating system uses virtual memory, so what should we do if we want to manipulate memory directly?

Linux maps each device to files in the / dev/ directory through which we can directly manipulate the hardware, and memory is no exception. In Linux, memory settings are mapped to / dev/mem,root users who can directly manipulate memory by reading and writing to this file.

The JVM process takes up too much virtual memory

When using TOP to view system performance, we will find that in the VIRT column, the Java process takes up a lot of virtual memory.

The reason for this problem is that Java uses Glibc's Arena memory pool to allocate a large amount of virtual memory and does not use it. In addition, the files read by Java are also mapped to virtual memory, and Java takes up 1m of virtual memory per thread stack under the default configuration of the virtual machine. You can see why multithreaded programs under linux consume virtual memory so much.

The actual physical memory consumption depends on the RES (resident) column, whose value is really mapped to the size of physical memory.

Common management commands

We can also manage the virtual memory of Linux ourselves.

View system memory status

There are many ways to check the memory of the system, free, vmstat and other commands can output the current memory status of the system, it should be noted that the available memory is not just free this column, due to the lazy feature of the operating system, a large number of buffer/cache will not be cleaned immediately after the process is no longer in use, if the process that used them can continue to be used again, they can also be used when necessary.

In addition, through cat / proc/meminfo, you can view the details of the system memory usage, including the status of dirty pages, and so on. For more information, please see: / PROC/MEMINFO Mystery.

Pmap

If you want to see the virtual memory distribution of a process separately, you can use the pmap pid command, which lists the occupancy of each segment of virtual memory from low address to high address.

You can add the-XX parameter to output more detailed information.

Modify memory configuration

We can also modify the system configuration of Linux by using sysctl vm [- options] CONFIG or directly reading and writing files in the / proc/sys/vm/ directory to view and modify the configuration.

SWAP operation

The SWAP feature of virtual memory is not always beneficial, allowing processes to exchange data between memory and disk will greatly take up CPU and reduce the efficiency of the system, so sometimes we don't want to use swap.

We can modify vm.swappiness=0 to set memory to use as little swap as possible, or simply use the swapoff command to disable SWAP.

The above is about the content of this article on "what is Linux virtual memory". I believe we all have a certain understanding. I hope the content shared by the editor will be helpful to you. If you want to know more about the relevant knowledge, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.