Shulou(Shulou.com)05/31 Report--

This article is to share with you about how to improve database performance in HugePages, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

Memory is an important resource for computers. Although most services today do not have such a high demand for memory, services such as databases and Hadoop buckets consume large amounts of memory. In production environments, they often take up memory of the order of GB and TB to improve computing speed. The Linux operating system has introduced a lot of strategies to better and faster manage these memories and reduce overhead. What we are going to introduce today is HugePages. That is, the big page [^ 1].

Most CPU architectures support larger pages, but different operating systems use different terms, such as HugePages on Linux, SuperPages on BSD, and LargePages on Windows, which represent similar large page functions.

Figure 1-CPU architecture and larger pages

We all know that Linux manages memory on a page-by-page basis, and the default page size is 4KB. Although some processors use 8KB and 16KB 64KB as the default page size, 4KB is still the mainstream of the operating system's default page configuration [^ 2]. Although the page size of 64KB is 16 times that of 4KB, the page size of 64KB is not large enough compared with the HugePages of the smallest 2MB, let alone the default 4KB:

Figure 2-default and large page size

2MB is generally the default size of HugePages, and even supports large pages of 1GB on the architecture of arm64 and x86Secret64, which is 262144 times the default page size of Linux. We can use the command shown below to view the relevant information of HugePages on the current machine:

$cat / proc/meminfo | grep Huge AnonHugePages: 71680 kB ShmemHugePages: 0 kB FileHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB

From the above output, we can see that the default size of large pages on the current machine is 2MB and the number of large pages is 0, that is, no process is applying for or using large pages. Readers can try to execute the above command on Linux, and if no additional configuration has been made on the machine, the output from using the above command will not be much different from that here.

The data stored in / proc/sys/vm/nr_hugepages is the number of large pages. Although its value is 0 by default, we can apply for or release large pages in the operating system by changing the contents of the file:

$echo 1 > / proc/sys/vm/nr_hugepages $cat / proc/meminfo | grep HugePages_ HugePages_Total: 1 HugePages_Free: 1.

In Linux, in the same way as other memory requests and releases, we can pass a MAP_HUGETLB tag to the mmap system call to apply for a large page of the operating system and use munmap to free memory [^ 3]. Use the code snippet shown below to apply for a large page of 2MB in the operating system:

Size_t s = (2UL * 1024 * 1024); char * m = mmap (NULL, s, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB / * flags * /,-1,0); munmap (m, s)

Although the application method of HugePages is not much different from the default memory, it is actually a special resource managed separately by the operating system. Linux will display the relevant data of HugePages separately in / proc/meminfo, and the container orchestration system Kubernetes will also think that large pages are independent resources different from memory. Pod shown below also needs to apply for large page resources separately [^ 4]:

ApiVersion: v1 kind: Pod metadata: name: huge-pages-example spec: containers:-name: example. VolumeMounts:-mountPath: / hugepages-2Mi name: hugepage-2mi-mountPath: / hugepages-1Gi name: hugepage-1gi resources: limits: hugepages-2Mi: 100Mi hugepages-1Gi: 2Gi memory: 100Mi requests: memory: 100Mi volumes:-name: hugepage-2mi emptyDir: medium: HugePages-2Mi-name: hugepage-1gi emptyDir: medium: HugePages-1Gi

As a new feature introduced by Linux from 2.6.32, HugePages can improve the performance of memory-intensive services such as databases and Hadoop home buckets. This feature is not very helpful to common Web services and back-end services, but may affect service performance. In this article, we will explain why HugePages can improve the performance of databases and other services:

HugePages can reduce the management overhead of memory pages.

HugePages can lock memory and prohibit memory swapping and freeing in the operating system.

Administrative overhead

Although the opening of HugePages mostly requires additional configuration from developers or operations engineers, enabling HugePages in an application can reduce the overhead of managing memory pages in the following ways:

Larger memory pages can reduce the level of page tables in memory, which can not only reduce the memory footprint of page tables, but also reduce the performance loss of converting from virtual memory to physical memory.

Larger memory pages mean higher cache hit rates, and CPU has a higher chance of getting the corresponding physical address directly from TLB (Translation lookaside buffer).

Larger memory pages can reduce the number of times to get large memory. Using HugePages can get the memory of 2MB each time, which is 512 times the default page efficiency of 4KB.

Because the address space of the process is virtual, CPU and the operating system need to record the corresponding relationship between the page and the process. The more pages in the operating system, the more time we need to find the physical memory corresponding to the virtual memory in the five-tier page table structure shown below. We will access the directory (Directory) in the page table according to the virtual address and finally find the corresponding physical memory:

Figure 3-five-layer page table for the default page

As shown in the figure above, if we use the default 4KB memory page in Linux, CPU needs to read PGD, PUD, PMD and PTE to obtain physical memory when accessing the corresponding memory, but the large memory of 2MB can reduce the number of directory accesses:

Figure 4-Page Table and large Page

Because the memory pages of 2MB occupy 21-bit addresses, we no longer need the PTE structure in the five-tier page table, which can not only reduce the number of times to access the page table when translating virtual addresses, but also reduce the memory footprint of the page table.

CPU can always find the physical page corresponding to the virtual page through the above complex directory structure, but it is very expensive to use the above structure every time the virtual address is translated. The operating system uses TLB as the cache to solve this problem. TLB is a part of the memory management component (Memory Management Unit), in which the cached page table entries can help us quickly translate the virtual address:

Figure 5-TLB

A larger memory page means a higher cache hit ratio, because the capacity of the TLB cache is certain, and it can only cache a specified number of pages. In this case, caching large pages of 2MB can improve the cache hit ratio for the system, thus improving the overall performance of the system.

In addition to fewer page table items and improved cache hit ratio, using larger pages can also improve memory access efficiency. For the same 1GB memory, memory pages using 4KB require 262144 processing times, but large pages using 2MB only need 512 times, which can reduce the number of processing required by the system to obtain memory by several orders of magnitude.

Lock memory

Using HugePages, memory can be locked and memory swapping and freeing of the operating system is prohibited. The Linux system provides a swap partition (Swap) mechanism, which copies a part of memory pages from memory to disk when there is insufficient memory, frees up memory space occupied by memory pages, and then swaps to memory when the corresponding memory process is accessed. This mechanism can create a false appearance of sufficient memory for the process, but it will also cause various problems.

Figure 6-swap Partition

In the article why NUMA will affect the latency of the program, we have introduced that Swap may affect the performance of the database when opening NUMA [^ 5]. Occasional Swap in the system is not unacceptable, but reading and writing disks frequently will significantly slow down the operating system.

Unlike other memory pages, HugePages is pre-allocated by system engineers using commands on the operating system, and when processes request large pages through mmap or other system calls, they get pre-allocated resources. The HugePages in Linux is locked in memory, so even when the system is out of memory, they will not be Swap to disk, which eliminates the possibility of important memory being swapped in and out frequently [^ 6].

REHL 6 introduces transparent large pages (Transparent Huge Pages, THP), which is an abstract layer that can automatically create, manage, and use large pages, hiding the complexity of using large pages for system administrators and developers, but it is not recommended to open in databases and similar loads. [^ 7]

The above is how to improve database performance in HugePages. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.