Shulou(Shulou.com)06/01 Report--

This article mainly shows you the "sample analysis of 52-bit virtual address support in the ARM64 kernel", which is easy to understand and well-organized. I hope it can help you solve your doubts. Let the editor lead you to study and study the "sample analysis of 52-bit virtual address support in the ARM64 kernel".

With the introduction of 64-bit hardware, the need to handle larger address space has increased.

When 64-bit hardware becomes available, the need to handle a larger address space (larger than 232 bytes) becomes obvious. Today, some companies already offer servers with 64TiB or more memory, and the x86 addresses 64 architecture and arm64 architecture now allow addressing of more than 248bytes of address space (the default 48-bit address support can be used).

The x86 pages 64 architecture supports these use cases by enabling hardware and software to enable five-level page tables. The address space that it allows for addressing is equal to 257 bytes (see x86: enabling level 5 page tables in the 4.12 kernel for details). It breaks through the upper limit of virtual address space 128PiB and physical address space 4PiB.

The arm64 architecture achieves the same functionality by introducing two new architectures, ARMv8.2 LVA (greater virtual addressing) and ARMv8.2 LPA (greater physical address addressing). This allows the use of the virtual address space of 4PiB and the physical address space of 4PiB (that is, 252 bits respectively).

As the ARMv8.2 architecture extension is supported in the new arm64 CPU, open source software now supports these two new hardware extensions.

Starting with the Linux 5.4 kernel, 52-bit (large) virtual address (VA) and physical address (PA) in the arm64 architecture are supported. Although the kernel documentation describes these features and the impact of the new kernel runtime on the old CPU (which does not support 52-bit virtual address extension at the hardware level) and the new CPU (the hardware level supports 52-bit virtual address extension), it can be complicated for the average user to understand these and how to "choose" the 52-bit address space.

Therefore, I will introduce the following relatively new concepts in this article:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

After adding support for these functions, how the memory layout of the kernel is "flipped" to the Arm64 architecture

Impact on user-mode applications, especially on programs that provide debugging support (e.g. kexec-tools, makedumpfile, and crash-utility)

How to use the household application "Select" to accept VA from a 52-bit address space by specifying a mmap parameter greater than 48 bits?

LVA and LPA extension of ARMv8.2 Architecture

The ARMv8.2 architecture provides two important extensions: large virtual addressing (LVA) and large physical addressing (LPA).

When 64 KB translation granularity is used, ARMv8.2-LVA provides a larger 52-bit virtual address space for each translation table base address register.

Allow in ARMv8.2-LVA:

When 64 KB translation granularity is used, the intermediate physical address (IPA) and physical address space are expanded to 52 bits.

If 64 KB translation granularity is used to support 52-bit physical addresses, then the first-level block will cover the address space of 4TB.

It is important to note that these features are only supported in the AArch74 architecture.

The following Arm64 Cortex-A processors currently support ARMv8.2 extensions:

Cortex-A55

Cortex-A75

Cortex-A76

Please refer to the Armv8 Architecture reference Manual for more details.

Kernel memory layout of Arm64

As the ARMv8.2 extension adds support for LVA addresses (available only when running with a page size of 64 KB), the number of descriptors increases in the first level of translation.

The user address sets the 63-48 bits to 0, while the kernel address sets these bits to 1. The choice of TTBRx is determined by the 63 bits of the virtual address. Swapper_pg_dir contains only kernel (global) mappings, while pgd contains only user (non-global) mappings. The swapper_pg_dir address is written to TTBR1 and never to TTBR0.

The Linux memory layout under the AArch74 architecture with a page size of 64 KB and three levels (with 52-bit hardware support) is as follows:

Start and end size use-0000000000000000 000fffffffffffff 4PB User fff0000000000000 fff7ffffffffffff 2PB kernel logic memory mapping fff8000000000000 fffd9fffffffffff 1440TB [gap] fffda00000000000 ffff9fffffffffff 512TB Kasan shadow area ffffa00000000000 ffffa00007ffffff 128MB bpf jit area ffffa00008000000 ffffa0000fffffff 128MB module ffffa00010000000 fffff81ffffeffff ~ 88TB vmalloc area fffff81fffff0000 fffffc1ffe58ffff ~ 3TB [protected area] fffffc1ffe590000 fffffc1ffe9fffff 4544KB fixed mapping fffffc1ffea00000 fffffc1ffebfffff 2MB [protected area] fffffc1ffec00000 fffffc1fffbfffff 16MB PCI I pico space fffffc1fffc00000 fffffc1fffdfffff 2MB [protected area] fffffc1fffe00000 ffffffffffdfffff 3968GB vmemmap ffffffffffe00000 ffffffffffffffff 2MB [protected area]

4. The transformation query table of the KB page is as follows:

+-+ | 63 56 | 55 48 | 47 40 | 39 32 | 31 24 | 23 16 | 15 8 | 7 0 | + + -+ | | v | | | | [11:0] intra-page offset | +-> [20:12] L3 index | +-> [29:21] L2 index | | +-- " -> [38:30] L1 index | +-> [47:39] L0 index +- -> [63] TTBR0/1

The conversion query table for the 64 KB page is as follows:

+-+ | 63 56 | 55 48 | 47 40 | 39 32 | 31 24 | 23 16 | 15 8 | 7 0 | + + -+ | v | | | [15:0] intra-page offset | +-> [28:16] L3 index | | +-> [41:29] L2 index | +-- " -- > [47:42] L1 index (48 bits) | [51:42] L1 index (52 bits) +- -> [63] TTBR0/1

Arm64 Multi-level Translation

Kernel support for 52-bit virtual addresses

Because newer kernels that support LVA should work on both the old CPU (the hardware does not support LVA extensions) and the new CPU (hardware supports LVA extensions), the design approach is to use a single binary to support 52 bits (if the hardware does not support this feature, you must be able to fallback to 48 bits at the beginning of boot). In other words, the 52-bit virtual address and the fixed-size PAGE_OFFSET,VMEMMAP must be set large enough.

This design requires the kernel to support the following variables for the new virtual address space:

VA_BITS constant * maximum * virtual address space size vabits_actual variable * actual * virtual address space size

Therefore, although VA_BITS sets the maximum virtual address space size, the actual supported virtual address space size is determined by vabits_actual (depending on the switch at startup).

Flip kernel memory layout

The design method of maintaining a single kernel binary requires that the .text of the kernel be in the high-order address, so they are the same for the 48amp 52-bit virtual address. Because the kernel address detector (KASAN) area accounts for only a small portion of the entire kernel virtual address space, for 48-bit or 52-bit virtual address space, the end of the KASAN region must also be in the upper half of the kernel virtual address space. (switch from 48 bits to 52 bits, the end of the KASAN area is unchanged and depends on ~ 0UL, while the starting address will "grow" to the lower address)

To optimize phys_to_virt () and virt_to_phys (), the page offset is maintained at 0xFFF0000000000000 (corresponding to 52 bits), which eliminates the need to read additional variables. Physvirt and vmemmap offsets will be calculated at early startup to enable this logic.

Consider the following physical and virtual RAM address space translation:

/ * the kernel linear address starts at the bottom of the virtual address space * the highest bit at the beginning of the test area is already a sufficient check And avoid the trouble of worrying about tags * / # define virt_to_phys (addr) ({\ if (! ((U64) addr) & BIT (vabits_actual-1)\ (addr) & ~ PAGE_OFFSET) + PHYS_OFFSET)}) # define phys_to_virt (addr) ( (unsigned long) ((addr)-PHYS_OFFSET) | PAGE_OFFSET) in the above code: the starting position of the PAGE_OFFSET-linearly mapped virtual address is in the TTBR1 address space PHYS_OFFSET-the starting position of the physical address and the effect of vabits_actual-* actual * virtual address space size on the user-mode program used to debug the kernel.

There are several user-space applications that can be used to debug a running / active kernel or to analyze vmcore dumps in the event of a system crash (for example, to determine the root cause of a kernel crash): kexec-tools, makedumpfile, and crash-utility.

When you use them to debug the Arm64 kernel, they are also affected because the Arm64 kernel memory mapping is "flipped". These applications also need to traverse the translation table to determine the physical address corresponding to the virtual address (similar to how it is done in the kernel).

Accordingly, after introducing "flip" into the kernel memory mapping, because the upstream destroys the user-mode application, it must be modified.

I have proposed fixes for three affected user-mode applications; some have been accepted upstream, but others are still waiting:

Propose upstream fixes for makedumpfile

Propose upstream fixes for kexec-tools

Fixes of accepted crash-utility

Unless these changes are made in user space applications, they will still not be able to debug the running / active kernel or analyze vmcore dumps in the event of a system crash.

52-bit user virtual address

To maintain compatibility with user-space applications with a maximum of 48 bits that rely on the ARMv8.0 virtual address space, the kernel returns virtual addresses from the 48-bit range to user space by default.

By specifying mmap prompt parameters greater than 48 bits, the user program can "choose" to receive virtual addresses from the 52-bit space.

For example:

.mmap _ high_addr.c---- maybe_high_address = mmap (~ 0UL, size, prot, flags,...)

You can also build a debug kernel that returns addresses from 52-bit space by enabling the following kernel configuration options:

CONFIG_EXPERT=y & & CONFIG_ARM64_FORCE_52BIT=y Please note that this option is for debugging applications only and should not be used in actual production.

The above is all the contents of the article "sample Analysis of 52-bit Virtual address support in the ARM64 Kernel". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.