Shulou(Shulou.com)06/01 Report--

This article introduces how to design and implement the kubernetes container network interface midonet network plug-in. The content is very detailed. Interested friends can refer to it for reference. I hope it can help you.

What is CNI?

CNI (Container Network Interface) is an operational container network specification, including method specifications, parameter specifications, etc. CNI only cares about the container's network connections, allocates network resources when the container is created, and deletes the allocated resources when the container is deleted. Because of this focus, CNI has broad support and specifications that are easy to implement. The CNI interface only needs to implement two methods, one called when a container is created and one called when a container is deleted.

How Kubernetes supports and runs plug-ins compliant with CNI specifications

Kubernetes first completes the network resource setup of the container in the form of a plug-in (pod). Built-in plugins include:cni,kubenet,hostport, etc. Let's talk about Kubenet. This is a simple network plug-in that creates a br0 bridge on each machine and ip connects to the br0 bridge for each pod according to PodCIDR settings. The second method can be combined with some network routing tools to complete a small-scale cluster network pod interconnection. We mainly talk about CNI plug-ins. Kubernetes supports the CNI specification with CNI plug-ins, calling various network plug-ins developed by other vendors and individuals that follow the CNI specification, such as Calico,Flannel, etc. K8s By default, CNI mode does not support port mapping, etc. k8s leaves container network settings none to the plug-in to manage container network resources.

What are the network resources mentioned several times above?

Container network resources include virtual network cards,IP addresses, DNS, network routes, etc. Containers use separate network namespaces and can have their own network resource information. This information data is configured for containers by different CNI plug-ins based on different SDN network implementations.

MidoNet SDN Network

MidoNet is a network virtualization software developed by Midkura, a Japanese SDN company. It realizes network virtualization based on the underlying physical facilities. It has distributed, decentralized and multi-layered characteristics. It mainly serves as the default network component in OpenStack, allowing virtual network solutions, especially those designed for network infrastructure, to serve cloud platforms such as OpenStack and virtualize their network storage stacks. MidoNet assigns a logical router to each tenant. Tenants are isolated from each other. Tenants can communicate with each other. Midonet supports L2 switching, L3 routing and L4 Load Balancer.

Stateful and stateless NAT, logical and distributed firewalls, BGP and ECMP support. Its architecture mainly consists of the following components:

Midolman（Midonet Agent）：Midonet

Agent is installed in each computing node, responsible for establishing network traffic control and providing distributed Midonet network services, routing, NAT, etc. He stores relevant virtual network information in NSDB.

Network State

Database(NSDB): stores network configuration and status, network topology, routing, Midonet does not centrally process network functions, but is handled by Midonet Agent. Midonet Agent synchronizes with NSDBs in real time and updates NSDB in time when there are changes. Midonet supports large-scale SDN clusters, and its architecture theoretically supports tens of thousands of nodes. We can use MidoNet to complete the intra-tenant Pod network interconnection within the k8s cluster.

MidoNet Multitenant Network Architecture Model

SDN(Software Defined Network), Midonet software defines the network components you know. Here are a few core software-defined concepts:

Router

A tenant corresponds to a Router, and is connected to the Bridge network interworking of the same Router. Midonet creates a PrivierRouter, and all tenant Routers connect to the PrivierRouter and communicate with the external network. Equivalent to a router network interworking, connecting the superior router to access the public network.

Bridge

A tenant can have multiple bridges, each Bridge using a different segment. For example, a Bridge segment is 192.168.0.0/24, and a maximum of 253 virtual devices can be connected to this Bridge.

Port(device communication port)

The communication interface between Router and Router and between Router and Bridge.

Route

Routing rules, which define the rules for forwarding traffic packets to the Router.

Rule

Define packet filtering criteria. Similar to iptables.

Implementation of Kubernetes CNI Plug-in Based on MidoNet

Midonet data exchange works at three layers, but it does not provide IP address management (IPAM) itself, so the Midonet based cni plug-in needs to complete the following tasks: IPAM, tenant Router, Bridge creation, container network card creation, and all end-to-end connections and route filtering rules creation.

IPAM

Two levels of IP management need to be completed, Router level address management, each Router has an IP address, and the global unique does not conflict. Each Bridge has a unique network segment, and connected virtual network cards have globally unique IPs.

IPAM can be implemented in many ways. CNI plug-ins are stateless applications. Perhaps you need a daemon to do IPAM work. Based on the idea of simplifying the architecture, we use ETCD to store IP data, and the plug-in directly manipulates ETCD. Complete IP usage and release.

tenant network initialization

When a new tenant creates a container for the first time, the tenant virtual appliance is initially created. We have already introduced what virtual appliances a tenant needs to create. Here I will talk about the details. Midonet provides a Rest-API to operate virtual appliances. Note here that different versions of the API are used depending on which version of Midonet is used. https://github.com/barnettZQG/golang-midonetclient

Encapsulated golang version of Midonet API operation method, support 1.* and 5.* API version. The creation steps are as follows:

Create tenant, call Keystone API.

Create Router and include creating in and out Chain.

Create PrivierRouter Port and assign IP, create Router Port and assign IP. Create a PortLink to connect the two ports.

Create routing rules for the Chain created earlier

Create packet filtering rules for the ports created earlier

Create a default Bridge. And create a Port to connect to the Router.

Store the data created above in ETCD.

Container NIC Creation and Network Binding Virtual Ethernet Pair

A veth pair, for short, is a pair of ports, and all packets coming in from one end of the pair will come out the other, and vice versa. Both ends can exist in different network namespaces. After the container is successfully created, there is a network space. At this time, k8s calls the ADD method of CNI plug-in to set the network. The plugin first creates a pair of Veth pairs. Place one end in the host network space and call the Midonet Binding API to bind it to a Bridge Port. The other end is inside the container and given an IP address, based on the Bridge segment currently in use. Docker0 network card part of the same principle.

Set up routing rules within containers

Set the default route to the NIC created above. For example, the network card created above is named eth0.

set the DNS

Set up some DNS information as needed.

How do you do that?

1. Use the shell command.

ip link *ip netns *ip address *ip route *

The above command details the use of a lot of online methods, no longer described here.

golang netlink library

https://github.com/vishvananda/netlink defines user-space interfaces related to network cards that communicate with the linux kernel.

CNI plug-in implementation considerations

CNI plug-ins add and delete operations should be idempotent, that is, the same parameter should have the same effect no matter how many times it is called.

CNI plug-ins should support concurrency, mainly the creation of tenant-related components and strong consistency in IP address allocation.

About how to carry out kubernetes container network interface midonet network plug-in design and implementation to share here, I hope the above content can have some help for everyone, you can learn more knowledge. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.