Shulou(Shulou.com)05/31 Report--

How to understand the principle of HugePages, in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Before introducing HugePages, let's review the relationship between virtual memory and physical memory under Linux.

Physical memory: that is, a stick of memory installed in a computer, such as a strip of memory the size of 2GB, then the physical memory address ranges from 0 to 2GB.

Virtual memory: the virtual memory address. Because CPU can only use physical memory addresses, it is necessary to convert virtual memory addresses to physical memory addresses before they can be used by CPU. This conversion process is done by MMU (Memory Management Unit, memory management unit). In a 32-bit operating system, the size of virtual memory space is 0 ~ 4GB.

We use figure 1 to describe the process of converting a virtual memory address to a physical memory address:

As shown in figure 1, the page table holds the mapping between the virtual memory address and the physical memory address. MMU finds the physical memory address mapped by the virtual memory address from the page table, and then submits the physical memory address to CPU, a process similar to the Hash algorithm.

Memory mapping is measured in memory pages. Typically, the size of a memory page is 4KB (shown in figure 1), so it is called a paging mechanism.

I. memory mapping

Let's take a look at the process of converting virtual memory addresses to physical memory addresses in a 64-bit Linux system (Intel x64 CPU), as shown in figure 2:

As you can see from figure 2, Linux uses only the first 48 bits (0-47 bits) of the 64-bit virtual memory address, and Linux divides the 48-bit virtual memory address into five parts, as follows:

PGD index: 39-47 bits (9 bits in total), specifying the index in the page global catalog (PGD,Page Global Directory).

PUD index: 30-38 bits (9 bits in total), specifying the index in the page parent directory (PUD,Page Upper Directory).

PMD index: 21-29 bits (9 bits in total), specifying the index in the middle page directory (PMD,Page Middle Directory).

PTE index: 12-20 bits (9 bits total), specifying the index in the page table (PT,Page Table).

Offset: 0-11 bits (12 bits in total), specifying the offset in the physical memory page.

The page table in figure 1 is divided into four levels: page global catalog, page parent catalog, middle page directory, and page table in order to reduce memory consumption (think about why memory consumption can be reduced).

Note: the page global catalog, page parent catalog, page middle catalog, and page table all occupy a 4KB-sized physical memory page. Because 64-bit memory addresses occupy 8 bytes, a 4KB-sized physical memory page can hold 512 64-bit memory addresses.

In addition, CPU has a register called CR3, which holds the starting physical memory address of the page's global catalog (as shown in figure 2). Therefore, the process of converting a virtual memory address to a physical memory address is as follows:

The physical memory address of the page global catalog is obtained from the CR3 register, and then the physical memory address of the page parent catalog is read from the page global catalog with 39 ~ 47 bits of the virtual memory address as the index.

With 30-38 bits of the virtual memory address as the index, the physical memory address of the middle directory of the page is read from the parent directory of the page.

Taking the 21-29 bits of the virtual memory address as the index, the physical memory address of the page table is read from the middle directory of the page.

With 12-20 bits of the virtual memory address as the index, the physical memory address of the physical memory page is read from the page table.

Taking the 0 ~ 11 bits of the virtual memory address as the offset of the physical memory page, the final physical memory address is obtained.

II. HugePages principle

It was described above that 4KB's memory pages are used as memory mapping units, but there are scenarios where we want to use larger memory pages as mapping units (such as 2MB). Using larger memory pages as mapping units has the following benefits:

Reduce the failure of TLB (Translation Lookaside Buffer).

Reduce the memory consumption of page tables.

Reduce the number of PageFault (missing page interruptions).

Tips:TLB is a cache, and TLB caches the virtual memory address and the physical memory address it maps. MMU first looks up the memory-mapped relationship from TLB, and if it finds it, it doesn't have to look up the page table backwards. Otherwise, you can only look up the mapped physical memory address in the page table based on the virtual memory address.

Because the larger the memory page that is mapped, the smaller the page table is required (which is easy to understand); the smaller the page table, the less TLB failures will occur.

The mechanism of using memory pages larger than 4KB as memory mapping units is called HugePages. At present, the commonly used HugePages sizes in Linux are 2MB and 1GB. Let's take memory pages with 2MB size as an example.

To map larger memory pages, you only need to increase the offset, as shown in figure 3:

As shown in figure 3, the offset portion is now extended to 21 bits (the page table part is overwritten, and the 21 bits can represent a size range of 0-2MB), so the middle directory points directly to the mapped physical memory page address.

In this way, the memory consumption of the page table portion can be reduced. Due to fewer memory mapping relationships, TLB failures are also reduced.

III. Use of HugePages

Now that we understand how HugePages works, let's introduce how to use HugePages.

The use of HugePages is not as simple as an ordinary memory request, but needs to be created with the help of the Hugetlb file system. The following steps for using HugePages are described:

1. Mount the Hugetlb file system

The Hugetlb file system is created specifically for HugePages, and we can mount an Hugetlb file system with the following command:

$mkdir / mnt/huge $mount none / mnt/huge-t hugetlbfs

After executing the above command, we mounted the Hugetlb file system in the / mnt/huge directory.

two。 Declare the number of available HugePages

To use HugePages, first declare to the kernel the number of HugePages that can be used. The / proc/sys/vm/nr_hugepages file holds the number of HugePages that the kernel can use, and we can set the new number of available HugePages using the following command:

$echo 20 > / proc/sys/vm/nr_hugepages

The above command sets the number of HugePages available to 20 (that is, 20 2MB memory pages).

3. Write the code to apply for HugePages

To use HugePages, you must use mmap system calls to map virtual memory to files in the Hugetlb file system, as shown in the following code:

# include # define MAP_LENGTH (101024024) / / 10MB int main () {int fd; void * addr; / / 1. Create a file for the Hugetlb file system fd = open ("/ mnt/huge/hugepage1", O_CREAT | O_RDWR); if (fd < 0) {perror ("open ()"); return-1;} / / 2. Map virtual memory to files in the Hugetlb file system addr = mmap (0, MAP_LENGTH, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); if (addr = = MAP_FAILED) {perror ("mmap ()"); close (fd); unlink ("/ mnt/huge/hugepage1"); return-1;} strcpy (addr, "This is HugePages example...") Printf ("% s\ n", addr); / / 3. After use, unmap munmap (addr, MAP_LENGTH); close (fd); 35 unlink ("/ mnt/huge/hugepage1"); 36 37 return 0; 38}

Compile the above code and execute it, and if there is no problem, the following information will be output:

This is HugePages example... The answer to the question on how to understand the principle of HugePages is shared here. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.