Shulou(Shulou.com)06/01 Report--

This article focuses on "how to understand the high-end memory under the Linux system". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to understand the high-end memory under the Linux system.

Linux kernel address space partition

Typically, the 32-bit Linux kernel virtual address space is divided into user space (3G) and kernel space (note that only 1G linear addresses can be used by the kernel). Note that this is the 32-bit kernel address space partition, while the 64-bit kernel address space partition is different.

Typically, the 32-bit Linux kernel virtual address space is divided into user space (3G) and kernel space (note that only 1G linear addresses can be used by the kernel). Note that this is the 32-bit kernel address space partition, while the 64-bit kernel address space partition is different.

The Origin of High-end memory in Linux Kernel

When kernel module code or thread accesses memory, the memory addresses in the code are logical addresses, and corresponding to the real physical memory addresses, one-to-one mapping of addresses is required. For example, the physical address corresponding to the logical address 0xc0000003 is 0 × 3, and the physical address corresponding to 0xc0000004 is 0 × 4,... ... The relationship between the logical address and the physical address is

Physical address = logical address-0xC0000000: this is the address translation relationship of the kernel address space. Note that the virtual address of the kernel is at the "high end", but the physical memory address of the ta map is at the low end.

In fact, the "kernel direct mapping space" is less than 1G, and we have to leave some linear space for the "kernel dynamic mapping space".

Therefore, Linux specifies that the "kernel direct mapping space" maps up to 896m of physical memory.

For high-end memory, you can get the corresponding page through alloc_page () or other functions, but to access the actual physical memory, you have to convert the page to a linear address (why? Think about how MMU accesses physical memory, that is, we need to find a linear space for the page corresponding to high-end memory, a process called high-end memory mapping.

Assuming that according to the above simple address mapping relationship, then the kernel logical address space access is 0xc0000000 ~ 0xffffffff, then the corresponding physical memory range is 0 × 0 ~ 0 × 40000000, that is, only 1 GB of physical memory can be accessed. If 8 GB of physical memory is installed in the machine, the kernel can only access the first 1 G of physical memory, and the latter 7 GB of physical memory will not be accessible, because the address space of the kernel has been fully mapped to the physical memory address range of 0 × 0 ~ 0 × 40000000. Even if 8 gigabytes of physical memory is installed, how can the kernel access the memory with a physical address of 0 × 40000001? There must be a memory logical address in the code. The address space of 0xc0000000 ~ 0xffffffff has been used up, so the memory after the physical address 0 × 40000000 cannot be accessed.

Obviously, the kernel address space 0xc0000000 ~ 0xfffffff cannot be used entirely for simple address mapping. Therefore, the x86 architecture divides the kernel address space into three parts: ZONE_DMA, ZONE_NORMAL, and ZONE_HIGHMEM. ZONE_HIGHMEM is high-end memory, which is the origin of the concept of high-end memory.

In the x86 structure, the three types of areas (starting with 3G) are as follows:

16MB starting from ZONE_DMA memory

ZONE_NORMAL 16MB~896MB

ZONE_HIGHMEM 896MB ~ end (1G)

High-end memory refers to memory with a physical address greater than 896m. For such memory, it cannot be mapped in the "kernel direct mapping space".

Why?

Because the "kernel direct mapping space" can only map from 3G to 4G at most, and can only directly map 1G physical memory, there is nothing we can do for physical memory larger than 1G.

There are three ways to map high-end memory:

1. Mapping to "kernel dynamic mapping space"

This approach is simple, because with vmalloc (), when requesting memory in the "kernel dynamic mapping space", it is possible to get pages from the high-end memory (see the implementation of vmalloc), so it is possible that the high-end memory is mapped to the "kernel dynamic mapping space".

2. Permanent kernel mapping

If you get the page corresponding to the high-end memory through alloc_page (), how do you find a linear space for it?

The kernel sets aside a linear space for this purpose, from PKMAP_BASE to FIXADDR_START, to map high-end memory. On the2.4 kernel, this address range is from 4G-8M to 4G-4M. This space is called "kernel permanent mapping space" or "permanent kernel mapping space".

This space uses the same page catalog table as other spaces, which for the kernel is swapper_pg_dir, and for ordinary processes, it is pointed to through the CR3 register.

Typically, this space is 4m in size, so you only need a page table, which the kernel looks for by pkmap_page_table.

With kmap (), you can map a page to this space.

Because this space is 4m, it can map up to 1024 page at the same time. Therefore, for unused page, it should be released from this space in time (except for mapping is the solution), and a linear address corresponding to page can be released from this space through kunmap ().

3. Temporary mapping

The kernel reserves some linear space between FIXADDR_START and FIXADDR_TOP for special needs. This space is called "fixed mapping space".

Part of this space is used for temporary mapping of high-end memory.

This space has the following characteristics:

1. Each CPU takes up a piece of space

2. In the space occupied by each CPU, it is divided into several small spaces, each small space size is 1 page, and each small space is used for a purpose, which is defined in the km_type in kmap_types.h.

When you want to make a temporary mapping, you need to specify the purpose of the mapping. according to the purpose of the mapping, you can find the corresponding small space, and then use the address of this space as the mapping address. This means that a temporary mapping causes the previous mapping to be overwritten.

Temporary mapping can be achieved through kmap_atomic ().

The following figure simply shows how to map high-end memory

The linear address space of Linux memory is 4GB and is divided into two parts: the user space part (usually 3G) and the kernel space part (usually 1G). Here we focus on the kernel address space part.

The kernel manages all physical memory through the kernel page global catalog. Because the 3G space before the linear address is used by users, the first 768 items (just 3G) of the kernel page global catalog are all 0 except 0 and 1. The last 256 items (1G) are used to manage all physical memory. The kernel page global catalog is statically defined at compile time as an array of swapper_pg_dir, which starts at the physical memory address 0x101000.

As can be seen from the figure, the kernel linear address space starts from PAGE_OFFSET (usually defined as 3G). In order to load the kernel into memory, the 8m linear address from PAGE_OFFSET is used to map the physical memory address of the kernel (it can also be said that the virtual address of the kernel starts from PAGE_OFFSET). Then there is the mem_map array, where the initial linear address of mem_map is architecture-related. For example, for UMA architecture, since the 16m physical address space corresponding to the 16m linear address space starting from PAGE_OFFSET is the DMA area, the mem_map array usually starts with the linear address of PAGE_OFFSET+16M. The linear address space from PAGE_OFFSET to VMALLOC_START-VMALLOC_OFFSET is directly mapped to the physical memory space (one-to-one mapping, physical address linear address-PAGE_OFFSET). The size of this area is related to the actual physical memory size of the machine. Here VMALLOC_OFFSET is 8m on X86, which is mainly used to prevent out-of-boundary errors. On systems with relatively small memory, the remaining linear address space (minus white space, namely VMALLOC_OFFSET) is used by the vmalloc () function to map discontiguous physical address space to contiguous linear address space. On systems with relatively large memory, vmalloc () uses the linear address space from VMALLOC_START to VMALLOC_END (that is, PKMAP_BASE minus 2 pages of blank page size PAGE_SIZE (interpret VMALLOC_END)). At this point, the remaining linear address space (and minus 2 pages of blank space, namely VMALLOC_OFFSET) can be divided into two parts: the first part, from PKMAP_BASE to FIXADDR_START, is used by the kmap () function to establish a permanent mapping of high-end memory. The second part, from FIXADDR_START to FIXADDR_TOP, is a fixed-size temporary mapped linear address space (reference: Fixed virtual addresses are needed for subsystems that need to know the virtual address at compile time such as the APIC). On the X86 architecture, FIXADDR_TOP is statically defined as 0xFFFFE000, where the fixed-size space ends before the last 4K of the entire linear address space. This fixed-size space size is calculated at compile time and stored in the _ _ FIXADDR_SIZE variable.

It is the existence of the vmalloc () usage area, the kmap () usage area, and the fixed large cell (kmap_atomic () usage area) that limits the size of the ZONE_NORMAL area, and because the kernel needs these functions at runtime, it requires at least the size of VMALLOC_RESERVE in the linear address space. The size of VMALLOC_RESERVE is architecture-dependent, and on X86, VMALLOC_RESERVE is defined as 128m, which is why ZONE_NORMAL sizes are usually 16m to 896m.

At this point, I believe you have a deeper understanding of "how to understand the high-end memory under the Linux system". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.