Shulou(Shulou.com)05/31 Report--

This article introduces you how to analyze the I/O performance of Kata container, the content is very detailed, interested friends can refer to it, I hope it can help you.

All of the analysis below is based on Kata container version 1.6.2, and we will explore this topic to understand the pros and cons of using this technique in environments where I/O binding performance and security are critical requirements.

What is a Kata container?

Kata containers are lightweight VMs designed to integrate seamlessly with container orchestration software such as Docker and Kubernetes. One use case envisioned is running an untrusted workload, taking advantage of the additional isolation gained by not sharing the OS kernel with the host. However, in a recent survey of virtual machines and containers, the unquestioned assumption that using host kernels results in additional security was challenged. Kata is derived from Intel Clear Containers and Hyper RunV technologies. The goal of gVisor is to solve similar problems by filtering and redirecting system calls to a separate user-space kernel. As a result, gVisor suffers a runtime performance penalty. Further discussion of gVisor is beyond the scope of this blog.

Configure Kubernetes for Kata

Kata containers are OCI compliant, which means that container runtime interfaces (CRIs) that support external runtime classes can use Kata to run workloads. Examples of these CRIs currently include CRI-O and containerd, both of which use runC by default, but this can be replaced by kata qemu running. Starting with Kubernetes 1.14+, the RuntimeClass attribute flag has been upgraded to beta and is therefore enabled by default. Therefore, the setup is relatively simple.

Currently Kata supports qemu and firecracker hypervisor backends, but support for the latter is considered rudimentary, especially in the absence of host-to-client file sharing. This allows us to use kata qemu as the current option, where virtio-9p provides basic shared file system functionality, which is critical for analysis (the test path is the network file system installed on the host).

Without these prerequisites, Kata startup will fail silently (we hardly learned this).

This example bullet shows how to replace runC with the Kata runtime in a Minikube cluster. Note that at the time of writing, the Kata container has additional host requirements:

Kata will only run on machines configured to support nested virtualization.

Kata requires at least one Westmere processor architecture.

Without these prerequisites, Kata's will fail quietly (we learn from multiple practices).

For this analysis, a bare metal Kubernetes cluster was deployed, machines were configured with our appliances playbooks using OpenStack Heat, and they were configured as Kubernetes clusters using Kubespray. Kubespray supports other container runtime specifications besides Docker, such as CRI-O and containerd, which are necessary to support the Kata runtime.

Design I/O performance test scenarios

To benchmark the I/O performance of Kata containers, we presented equivalent scenarios for bare metal and runC containers to compare. In all cases, we used fio(version 3.1) as the I/O benchmark, called as follows, where $SCRATCH_DIR is the path to BeeGFS installed on the host (more on that later in this section):

fio fio_jobfile.fio --fallocate=none --runtime=30 --directory=$SCRATCH_DIR --output-format=json+ --blocksize=65536 --output=65536.json

The fio_jobfile.fio file referenced above reads as follows:

[global] ; Parameters common to all test environments ; Ensure that jobs run for a specified time limit, not I/O quantity time_based=1 ; To model application load at greater scale, each test client will maintain ; a number of concurrent I/Os. ioengine=libaio iodepth=8 ; Note: these two settings are mutually exclusive ;(and may not apply for Windows test clients) direct=1 buffered=0 ; Set a number of workers on this client thread=0 numjobs=4 group_reporting=1 ; Each file for each job thread is this size filesize=32g size=32g filename_format=$jobnum.dat [fio-job] ; FIO_RW is read, write, randomread or randomwrite rw=${FIO_RW}Scenario Number of Clients Disk I/O Mode Bare Metal 1 Sequential Read runC Container 8 Random Read Kata Container 64 Sequential Write Random Write

The parameter space explored for the I/O performance study covers a combination of 36 scenarios, number of clients, and disk I/O modes.

results

Disk I/O throughput

In these results, we plot the total bandwidth across all clients, showing the scale-up bandwidth that a single client can achieve and the scale-out throughput that can be achieved across many clients.

Disk I/O bandwidth comparison between bare metal, runC and Kata. In all cases, the bandwidth achieved with runC containers is slightly lower than bare metal. However, Kata containers generally perform much worse, getting about 15% bare metal read bandwidth when there are 64 clients, and a much smaller percentage of random write bandwidth. The only exception was sequential writes with 64 clients, where Kata containers performed about 25 percent better than bare metal scenarios.

Cumulative Distribution Function of Submission Delay (CDF)

In latency-sensitive workloads, I/O latency may dominate. I/O operation commit latency is plotted on a logarithmic scale to accommodate a very wide range of data points.

Comparison of CDF commit latency between bare metal, runC and Kata container environments for 1, 8 and 64 clients, respectively. There is a slight difference between running fio jobs in bare metal versus running them as runC containers. However, comparing bare metal to Kata containers, the overhead is significant in all cases.

Number of clients >1864ModeScenario50%99%50%99%50%99%sequential readbare15812670241633781453247095runC20072506239139071506246022Kata41124620126484646486409563806random readbare9702342258033051493543884runC11552277250638561537842229Kata547265861351731080109805314277sequential writebare101117282592150233730258834runC101119902547148924308233832Kata394848824102616014821190742random writebare1269202336981161619722159285runC1286195739281179619374151756Kata43585275456614254178055915343845

The table summarizes the 50% and 99% submission delays (in μs) corresponding to the numbers shown earlier. *

looking ahead

In this I/O-intensive scenario, Kata containers do not yet achieve the performance of traditional containers.

From the results, it is clear that there is a trade-off when choosing between bare metal, runC containers, and Kata containers. Although runC containers provide better consideration for most use cases, they still leave the host kernel vulnerable to exploitation of the system call interface as an attack surface. Kata containers provide isolation of hardware support, but there is currently a significant performance overhead, especially for disk I/O binding operations.

Kata's roadmap and pace of growth have solid foundations and optimistic prospects. The Kata team has recognized the performance pitfalls of using virtio-9p as a storage driver to share paths between host and guest VMs.

Kata version 1.7(to be released May 15, 2019) is expected to come with experimental support for virtio fs, which is expected to improve I/O performance issues. Initial results are encouraging, with other published benchmark reports showing virtio fs driver disk I/O bandwidth improvements of 2 to 8 times over virtio-9p. We will repeat our testing and analysis as new features become available.

About how to analyze the I/O performance of Kata container to share here, I hope the above content can be of some help to everyone, you can learn more knowledge. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.