Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to use RDMA in Kubernets. Many people may not know much about it. In order to let you know more, Xiaobian summarized the following contents for you. I hope you can gain something according to this article.

RDMA

RDMA(Remote Direct Memory Access) was created to address the latency of server-side data processing in network transmission.

The principle is to transfer data directly from the memory of one computer to the memory of another computer without the intervention of the operating system and protocol stack. By bypassing the operating system protocol stack, the communication process greatly reduces CPU usage, reduces kernel memory duplication, and context switching between user mode and kernel mode.

There are three common RDMA implementations: RDMA over Converged Ethernet, InfiniBand, and iWARP.

cdn.nlark.com/lark/0/2018/png/25353/1540806975240-659279da-052f-4607-8fa9-4212b9490ca2.png">

Alibaba Cloud supports RDMA

Alibaba Cloud supports SCC for supercomputing clusters and supports both RoCE networks and VPC networks, where RoCE networks are dedicated to RDMA communications. SCC is mainly used in high performance computing and artificial intelligence/machine learning, scientific/engineering computing, data analysis, audio and video Media Processing Service and other application scenarios.

RoCE (RDMA over Convergent Ethernet) network speeds reach the performance of InfiniBand networks and support a wider range of Ethernet-based applications.

Introduction: https://www.alibabacloud.com/help/zh/doc-detail/60576.htm

You can purchase SCC virtual machines directly from ECS console. https://www.alibabacloud.com/help/zh/doc-detail/61978.htm

Container Support RDMA

The Kubernetes Engine currently supports RDMA. You can add ECS of SCC model to the container cluster and support RDMA at the scheduling level through the Device plugin of RDMA.

By declaring resourcesLimit rdma/hca: 1, you can specify that containers are dispatched to ECs for RDMA.

Create a container cluster

Go to the Kubernetes Engine console and select Create Kubernetes Cluster. SCC is currently only supported in Shanghai. Please select East China 2 (Shanghai) as the container cluster region. After configuring other parameters, click Create Cluster and wait for the cluster to be created successfully.

Deploy RDMA Device plugin

In the Kubernetes Engine console, select Deploy with Templates. Deploy the RDMA enabled Device plugin and select the corresponding cluster and namespace. The template is as follows:

apiVersion: v1kind: ConfigMapmetadata: name: rdma-devices namespace: kube-systemdata: config.json: | { "mode">

Add SCC's ECS to the cluster by manual addition

Create an ECS for SCC and cluster it under the same VPC as the container. and join the same security group.

Operating clusters on the Kubernetes Engine console-> More-> Add existing ECS. Use Manual Add and select the RDMA machine created. Get Add Script.

Log on to ECS and execute the script. The final addition success is displayed as follows:

In Node-> Label Management, add the label aliyun.accelerator/rdma: true to the rdma node.

After adding the tag, you can see that the RDMA device plugin pod on the node has been successfully allocated to the SCC node.

Deploy two test images

apiVersion: v1kind: Podmetadata: name: rdma-test-podspec: restartPolicy: OnFailure containers: - image: mellanox/centos_7_4_mofed_4_2_1_2_0_0_60 name: mofed-test-ctr securityContext: capabilities: add: [ "IPC_LOCK" ] resources: limits: rdma/hca: 1 command: - sh - -c - | ls -l /dev/infiniband /sys/class/net sleep 1000000---apiVersion: v1kind: Podmetadata: name: rdma-test-pod-1spec: restartPolicy: OnFailure containers: - image: mellanox/centos_7_4_mofed_4_2_1_2_0_0_60 name: mofed-test-ctr securityContext: capabilities: add: [ "IPC_LOCK" ] resources: limits: rdma/hca: 1 command: - sh - -c - | ls -l /dev/infiniband /sys/class/net sleep 1000000

Executing ib\_read\_bw -q 30 in a container

After reading the above, do you know more about how to use RDMA in Kubernets? If you still want to know more knowledge or related content, please pay attention to the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.