Shulou(Shulou.com)06/02 Report--

Author | Ye Lei, senior technical expert of Alibaba

1. Kubernetes basic network model

This article introduces some of Kubernetes's ideas about the network model. As we all know, Kubernetes has no restrictions on the specific implementation of the network, and it does not give a particularly good reference case. Kubernetes limits whether a container network is qualified, that is, the container network model of Kubernetes. It can be summed up as three chapters and four goals.

The three chapters mean: when evaluating a container network or designing a container network, its access conditions. Which three items does it need to meet? It can be considered as a qualified network solution. The four major goals mean that when designing the topology of the network and the realization of the specific functions of the network, we should think clearly whether we can achieve these major indicators such as connectivity. draw up three chapters of law

First, let's take a look at the three chapters of the law:

The first: any two pod can actually communicate directly, without explicitly using NAT to receive data and address translation; second: node and pod can communicate directly without using obvious address translation; third: pod sees its own IP and others see it using the same IP, the middle can not be translated.

Later, I will talk about my personal understanding of why Kubernetes has some seemingly arbitrary models and requirements for container networks.

Four major goals

In fact, the four major goals are to design a K8s system to provide services to the outside world, from the perspective of the network, how can the outside world connect to the applications inside the container step by step?

How does the outside world communicate with service? That is, if there is an Internet or a user outside the company, how can I use service? Service specifically refers to the service concept in K8s. How does service communicate with its backend pod? How do calls between pod and pod communicate? Finally, there is the communication between containers within pod.

The ultimate goal is that the outside world can be connected to the innermost and provide services to the container.

Interpretation of basic constraints

The basic constraints can be interpreted as follows: because the complexity of the network development of the container is that it is actually parasitic on the Host network. From this point of view, container network solutions can be roughly divided into two Underlay/Overlay factions:

The standard of Underlay is that it is on the same layer as the Host network, and one of the features visible from the outside is whether it uses the same network segment of the Host network, input and output infrastructure, and whether the IP address of the container needs to cooperate with the Host network (from the same central distribution or unified division). This is what makes Underlay;Overlay different is that it does not need to apply for IP from the IPM management component of the Host network. Generally speaking, it only needs to be in conflict with the Host network, and the IP can be allocated freely.

Why did the community come up with such a simple and arbitrary model as perPodperIP? Personally, I think this has brought a lot of benefits for service to manage the tracking performance monitoring of some services. Because an IP is always good for case or all kinds of small things in the end.

2. Netns explores what Netns has achieved.

Let's talk a little bit about the kernel foundation that can be implemented on a network in Network Namespace. In a narrow sense, runC container technology does not rely on any hardware, its execution basis is its kernel, the kernel representative of the process is task, if it does not need isolation, then it uses the host space (namespace), and does not need a special space isolation data structure (nsproxy-namespace proxy).

On the contrary, if a separate network proxy, or mount proxy, should be filled with real private data. The data structure it can see is shown in the figure above.

From a sensory point of view, an isolated cyberspace will have its own network card or network device. The network card may be virtual or physical, and it will have its own IP address, IP table and routing table, and its own protocol stack state. This specifically refers to the TCP/Ip protocol stack, which will have its own status, iptables and ipvs.

In terms of the whole sense, this is equivalent to having a completely independent network, which is isolated from the host network. Of course, the code of the protocol stack is still common, but the data structure is different.

The relationship between Pod and Netns

This diagram can clearly show the relationship of Netns in pod. Each pod has its own cyberspace, and pod net container will share this cyberspace. Generally, K8s will recommend the Loopback interface to communicate between pod net container, and all container provide services through pod's IP. In addition, for the Root Netns on the host, you can think of it as a special cyberspace, but its Pid is 1.

III. A brief introduction to the mainstream network solutions typical container network implementation schemes

Next, we will briefly introduce the typical implementation of container network. Container network solution is probably the most blooming area in K8s, and it has a variety of implementations. In fact, the complexity of the container network lies in that it needs to coordinate with the underlying Iass layer network and make some choices in the flexibility of performance and IP allocation.

The following is a brief introduction to several major schemes: Flannel, Calico, Canal, and finally WeaveNet. Most of the intermediate schemes adopt a policy routing method similar to Calico.

Flannel is a relatively unified scheme, which provides a variety of network backend. Different backend implements different topologies, which can cover a variety of scenarios; Calico mainly uses policy routing, and nodes use BGP protocol to synchronize routes. It is characterized by rich functions, especially good support for Network Point. We all know that the requirements of Calico for the underlying network generally require mac addresses to be connected directly, not across layer 2 domains. Of course, there are also some community students who will integrate the advantages of Flannel and the advantages of Calico. We call it grafting innovative project Cilium; finally talk about WeaveNet, if you need to do some encryption of data in use, you can choose to use WeaveNet, its dynamic scheme can achieve better encryption. Flannel scheme

The Flannel scheme is the most commonly used at present. As shown in the figure above, you can see a typical container network scheme. The first thing it needs to solve is how the container packet arrives at Host, which is done by adding a Bridge. Its backend is actually independent, that is to say, how the package leaves the Host, which encapsulation method is used, or no encapsulation is required, are optional.

Now let's introduce the three main backend:

One is user-mode udp, which is the earliest implementation, and then the kernel's Vxlan, both of which are considered overlay's solutions. The performance of Vxlan will be better, but it requires the version of the kernel to support the function of Vxlan; if your cluster is not large enough and in the same layer 2 domain, you can also choose to use host-gw. Backend in this way is basically started by a broadcast routing rule with high performance. IV. The usefulness of Network Policy basic concepts of Network Policy

Let's introduce the concept of Network Policy.

I just mentioned that the basic model of Kubernetes network requires full interconnection between pod, which will bring some problems: maybe in a K8s cluster, there are some call chains that are not called directly. For example, between the two departments, then I hope that Department A will not visit the services of Department B. at this time, the concept of strategy can be used.

Basically, its idea is this: it uses various selectors (tags or namespace) to find a set of pod, or to find the equivalent of communication between the two ends, and then through the flow characteristics to determine whether they can be connected, can be understood as a whitelist mechanism.

Before using Network Policy, as shown in the figure above, note that apiserver needs to turn on these switches. Another more important thing is that the network plug-in we choose needs to support the landing of Network Policy. You should know that Network Policy is only an object provided by K8s, and there are no built-in components for implementation, it depends on whether the container network solution you choose supports this standard or not and the degree of completeness, if you choose the Flannel and so on, it does not really land this Policy, then it is useless for you to try this.

Configure an instance

Next, let's talk about an example of a configuration, or what should be done when designing a Network Policy? Personally, I think there are three things that need to be decided:

The first thing is to control the object, like the spec part of this instance. Through the selector of podSelector or namespace in spec, you can choose to do a specific set of pod to accept our control; the second is to consider the flow direction carefully, do you need to control the direction of entry or exit? Or do you have to control both directions? The most important thing is the third part, if you want to add a control object to the selected direction to describe its flow, which stream can be put in or out? Analogous to the quintuple of this flow feature, you can use some selectors to decide which ones can be my remote end, which is the choice of objects; you can also use the mechanism of IPBlock to get which IP can be released; and finally, which protocols or ports. In fact, the flow characteristics are summed up as a quintuple, which will select the specific acceptable streams. This paper summarizes

This is the end of this article, and let's briefly summarize it:

In pod's container network, the core concept is that IP,IP is the address basis of each pod's external communication, which must be consistent inside and outside, in line with the model characteristics of K8s.

Then when introducing the network scheme, the most important thing that affects the performance of the container network is the topology. To be able to understand how your packet is connected end-to-end, and how to get from container to Host,Host and whether the container should be encapsulated or unencapsulated? Or is it routed through policy? How did you solve it when you finally reached the opposite end?

Container network selection and design selection. If you are not clear about your external network, or if you need the most universal solution, assuming that you are not clear about whether the mac is directly connected or whether the routing table of the external router can be controlled, then you can choose Flannel to use Vxlan as the backend. If you are sure that your network is layer 2 directly connected, you can choose Calico or Flannel-Hostgw as a backend

Finally, there is Network Policy, which is a powerful tool for precise control of inflow and outflow when used in operation and maintenance. We also introduced the method of implementation, to figure out who you want to control, and then how to define your flow. Fifth, thinking time

Finally, leave some thoughts. You can think about it:

Why is the interface standardized CNI, but the container network does not have a very standard implementation, built into K8s?

Why doesn't Network Policy have a standard controller or a standard implementation, but instead give it to the owner of the container network to provide?

Is it possible to implement a container network without network equipment at all? Considering that there are RDMA and other schemes which are different from TCP/IP.

In the process of operation and maintenance, there are many network problems, and it is difficult to troubleshoot, so whether it is worthwhile to make an open source tool, so that it can friendly display from container to Host, Host to Host, or between encapsulation and unencapsulation, whether there are any problems in all stages of the network, can quickly locate. As far as I know, there should be no such tools now.

These are my basic concepts of K8s container network, as well as some introduction of Network Policy.

"Alibaba Yun × × icloudnative × × erverless, containers, Service Mesh and other technical fields, focusing on cloud native popular technology trends, cloud native large-scale landing practice, to do the best understanding of cloud native development × ×

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.