Shulou(Shulou.com)06/01 Report--

This article mainly introduces "the reason why the performance of virtualized Pod is better than bare metal". In daily operation, I believe many people have doubts about the reason why the performance of virtualized Pod is better than bare metal. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubt that "the performance of virtualized Pod is better than bare metal". Next, please follow the editor to study!

Why is the Native Pods of the Pacific Project faster?

Modern servers generally have multiple processors (CPU), using NUMA (non-uniform memory access) memory access mode. In the NUMA architecture, each CPU is responsible for managing a piece of memory called local memory.

When CPU accesses the memory managed by itself, it is relatively fast because it is the nearest access, but if you need to access the memory under other CPU names (called remote access), you often need to go through several circuit switches, which is usually slower.

When scheduling Pod, ESXi takes into account the locality (locality) of memory used by Pod and ensures that it accesses local memory as much as possible, so that Pod runs well and the overall CPU efficiency is improved. On the other hand, process schedulers in bare metal Linux may not be able to provide similar functionality between NUMA domains, so there is a performance penalty.

Knowing that Pod is a separate running entity, the ESXi CPU scheduler tries to ensure that its memory access is within the local NUMA domain, greatly reducing the number of remote memory accesses, thereby providing better performance for workloads in Pod and improving the overall efficiency of CPU. On the other hand, the process scheduler in Linux cannot well identify the differences between NUMA domains, so it cannot provide similar scheduling capabilities.

Performance Evaluation experiment of Pacific Project Native Pods

To compare performance, VMware engineers configured the test platform shown in figure 1 on the same hardware, each with 44 cores of 2.2 GHz and 512 GB memory:

A) ESXi nodes of two Pacific projects and their supervisor clusters

B) two mainstream enterprise Linux bare metal cluster nodes configured by default

Figure 1: test platform configuration

In general, a hyperthreaded processor core has multiple logical cores (hyperthreading) that share hardware resources. To reduce the impact on testing, hyper-threading is disabled in both test platforms. In each cluster, one node is used as the system under test (Worker Node), while Kubernetes Master is run on the other node.

Figure 2:Pod configuration

Ten Kubernetes Pod are deployed in the Worker node, each Pod has a resource limit of 8 CPU,42 GB memory, and a standard Java transaction benchmark is run in each container, as shown in figure 2.

Considering the complexity and nature of the workload used for us, a larger Pod was used in the experiment to manage test sample runs and Pod score summaries. Use the Pod definition to affinitized the Pod to the Worker nodes in each test platform. The total score (maximum throughput) of all 10 Pod is used to evaluate the performance of the system under test. There is basically no design of I / O or network transmission in the test, and all experiments are limited to a single Kubernetes node. Therefore, the impact of I / O or network performance is not discussed in this article.

Test result

Figure 3 shows the performance of a mainstream enterprise Linux bare metal node compared with that of the Pacific supervisor cluster (green bar), with bare metal Linux performance as a benchmark of 1.0.

The performance of the Pacific supervisor cluster is 8 per cent better than that of bare metal enterprise-class Linux.

Figure 3: relative performance of Pacific Supervisor Cluster and bare Metal Enterprise Linux Node

The test is repeated many times and the average is used to reduce the error of the experiment. Compared with the bare metal situation, the Pacific supervisor cluster can achieve an overall performance improvement of about 8%.

Analysis and optimization

Looking at the system statistics, compared to the vSphere supervisor cluster, the workload running on the bare metal is dragged down by many remote NUMA memory access. The performance advantage of the vSphere supervisor cluster mainly comes from the better CPU scheduling method, while offsetting the additional performance overhead caused by virtualization.

Further analysis shows that in the bare metal Linux, only about 43.5% of the missed L3 cache data can be obtained from the local DRAM, and the rest needs to be provided by remote memory. In contrast, the vSphere supervisor cluster benefits from the excellent CPU scheduling capability in ESXi, where 99.2% of the missed L3 data is available in the local DRAM, thus avoiding remote memory access and improving the performance of the vSphere supervisor cluster. (figure 4)

Figure DRAM hit ratio comparison between 4:vSphere supervisor cluster and bare metal Linux (the higher the number, the better)

To reduce the performance impact of non-local NUMA access on bare metal Linux, engineers tried some basic optimizations, such as switching NUMA balance switches and pinning to CPU using task set-based Pod, but these did not substantially improve performance. Currently, Kubernetes does not incorporate the CPU use of NUMA architecture into the Pod specification, so there is no good way to solve this problem for the time being.

The conclusion in this lab depends on the memory intensity of Pod access, and if the workload has different memory requirements, the impact of NUMA locality on its performance may be different. In short, Pod applications with high memory access frequency may perform better on vSphere supervisor clusters than on bare metal.

For more information, see:

Https://blogs.vmware.com/performance/2019/10/how-does-project-pacific-deliver-8-better-performance-than-bare-metal.html

At this point, the study on "the reason why virtualized Pod performance is better than bare metal" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.