Shulou(Shulou.com)06/01 Report--

How to match the value HugePages under Linux. Aiming at this problem, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

HugePages uses large pages of memory to replace traditional 4kb memory pages, which reduces the number of management virtual addresses, speeds up the mapping from virtual addresses to physical addresses, and improves the overall performance of memory by abandoning the swapping in and out of memory pages. Especially for memory above 8GB and larger Oracle SGA size, it is recommended to configure the value and use the HugePage feature.

Describe how to match the value of HugePages based on x86 / 64 Linux.

1. The introduction of Hugepage

It is much faster for operating system to access data directly from physical memory than to read and write data from disk, but physical memory is limited, which leads to the concept of physical memory and virtual memory. Virtual memory is a strategy proposed to meet the shortage of physical memory. It is a piece of logical memory virtual using disk space. This part of disk space is called virtual memory in Windows and swap space (Swap Space) in Linux.

For the management of this large memory (physical memory + virtual memory), most operating systems use segmented or paged management. Segmentation is a coarse-grained management mode, while paging is a fine-grained management mode, which can avoid the waste of memory space. Accordingly, there is the concept of physical address and virtual address of memory. In the previous two ways, CPU must translate the virtual address into the physical memory address in order to actually access the memory. To improve this conversion efficiency, CPU caches the mapping of the nearest virtual memory address to the physical memory address and stores it in a mapping table maintained by CPU. In order to improve the access speed of memory as much as possible, it is necessary to save as many mapping relationships as possible in the mapping table.

The memory management of linux adopts the paging access mechanism. In order to ensure that the physical memory can be fully utilized, the kernel will automatically swap the infrequently used pages in the physical memory to the virtual memory according to the LRU algorithm at the appropriate time, and keep the frequently used information to the physical memory. In general, Linux defaults to 4K per page, which means that if physical memory is large, there will be a large number of entries in the mapping table, which will affect the retrieval efficiency of CPU. Because the memory size is fixed, the only way to reduce the number of entries in the mapping table is to increase the page size. That's why Hugepage came. That is to break the traditional memory management of small pages, using large pages, 2m, 4m, 16m and so on. As a result, the number of mapping entries is significantly reduced. If the system has a large amount of physical memory (greater than 8 gigabytes), the physical 32-bit operating system or 64-bit operating system should use Hugepage.

II. Related concepts of hugepages

Page Table: page table is also an implementation for memory management for physical address to virtual mapping. Therefore, for memory access, first access to Page Table, and then according to the mapping in Page Table, implicitly transferred to the physical address to access data.

TLB: Translation Lookaside Buffer (TLB), a fixed-size cache in CPU that contains part of the page table mapping and is used to quickly translate virtual addresses into physical addresses.

Hugetlb: hugetlb is an entry in TLB that points to HugePage (usually greater than 4k or a predefined page size). HugePage is implemented through hugetlb entries, and it can also be understood that HugePage is a handle to hugetlb page entry.

Hugetlbfs: a new in-memory filesystem similar to tmpfs is proposed in kernel 2. 6.

III. Regular Pages and HugePages

A 、 Regular Pages

In the following figure, there are two different processes. The memory access of the two processes first accesses the local page table, and the local page table refers to the page of the system-wide table (that is, the TLB described earlier). Finally, the entry in the system-wide table points to the actual physical address. The physical address page size size 4kb in the figure. You can also see that both process 1 and process 2 point to page2, which is the same physical address, in system-wide table. The above situation occurs with the use of shared memory in Oracle sga.

B 、 Huge Pages

In the following figure, the huge page attribute is included in both local page table and system page table. So any page in the page table may use a regular page

It is also possible to use huge page. Similarly, both process 1 and process 2 share the Hpage2. The physical memory in the figure, the regular page size is 4kb, and the huge page size is 4mb.

IV. Advantages of hugepage

For large system memory and sga, using hugepage can greatly improve the performance of Oracle database.

A 、 Not swappable

There is no need to exchange. That is, there is no problem of swapping in and out of pages due to insufficient memory space.

B 、 Relief of TLB pressure

Reduce the pressure on TLB, that is, reduce the pressure on cpu cache cacheable address mapping. Due to the use of huge page, the number of virtual addresses managed is reduced at the same memory size.

TLB entry can contain more address space, and the addressing ability of cpu has been enhanced accordingly.

C 、 Decreased page table overhead

Reduce page table load. For ordinary page, each entry needs to be managed by 64bytes. For 50gb memory, managing these entry requires the size of 800mb.

(50,1024,1024) kb/4kb*64bytes/1024/1024=800mb.

D 、 Eliminated page table lookup overhead

Eliminate the load of page table lookup

E 、 Faster overall memory performance

Improve the overall performance of memory

About how to match the value HugePages under Linux. The answer to the question is shared here. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel to learn more about it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.