Shulou(Shulou.com)05/31 Report--

What this article shares with you is about the Mesos container network solution. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it with the editor.

Mesos is one of the main technology stacks of human cloud, and we have been practicing Mesos applications for a long time. From the perspective of multi-person cloud, we hope that the rapid distribution of containers can help customers achieve rapid delivery. Now that the Mesos community is growing very fast, Mesos has an important feature, Unified Container, that allows Mesos to deliver containers. Next, there is a hotter topic-"one container, one IP". How to solve this problem with network solutions in the current container circle, everyone shows their talents, but there are not many people who can really explain the network and containers clearly. Combined with the long-term accumulation and practice of several people, here I would like to discuss the container network and Mesos solutions to provide you with a reference.

# # Network problems with Mesos

In fact, Mesos 1.0 already has a complete container Interface specification. CNCF (Cloud Native Computing Foundation) provides the interface of Container Network Interface, hoping to regulate the use of the network in the Mesos community, so that everyone has an interface to reuse the network. Mesos 1.0, which has just been released, will encounter some problems. The most common is a container-IP that we cannot cross. Everyone has their own views on this issue, but the process of Mesos is sandbox, there is no concept of IP, so we have to think about how to get IP in the way of a container.

Second, with IP, there will be the problem of service discovery. What is service discovery? The simplest is DNS, a name for multiple IP. Third, with the network, there is a connection between all the data, how to isolate them is a problem. For example, if there is a database dedicated to business An and another database for business B, there are two Wordpress. There is no connection between these two multi-tenants. We hope that the network between them is isolated. Mesos has a solution that can be used immediately is the Calico project. Although there are commercial companies working on this project, its source code and ideas can be reused. So this time I am mainly to share with you the implementation of one container-IP.

# # Why choose Calico to implement one container and one IP?

} "title=" > A machine has its own MAC address, and there is a connection between its own IP,MAC address and IP on a regular host, and there is also a three-layer concept in it, but this is the network of the host, and there is also a MAC address after the container, but it is a virtual MAC address. How to get a real IP or the IP you want to assign to it leads to a concept-- L3 virtual network. It can be assigned a different MAC if there is no real IP address or if the MAC address is the same. Linux's Kernel has the concept of Filter, which can transform the data flow. We can make a judgment on the routing rules based on the integrity of Linux. If you use Linux Bridge mode, you can create IP. The source code of Calico project is based on this ability of Linux. The reason for the isolation between each container is that it can be divided into different network segments for each container, and the network segments are isolated. Originally, there are no routing rules between the two network segments and cannot communicate with each other.

Why is it important that Mesos has a CNI support? Before using Mesos and Doker, there are many ways to obtain IP, but these methods are very difficult to regulate in the Mesos ecosystem. Mesos has now removed Doker. For this part of the network, it needs a specification, which is the CNI specification. The CNI specification is a Json, this Json is very simple and a good semantics, and it can support IPV4 and later IPV6. It is enough for a configuration file to describe the entire network structure. With CNI, there are many implementations, and Calico is just one of them. You will see modes with hardware, Contive, Weave, Vxlan, and so on. In fact, there are Overlay patterns in them.

Mesos itself is a set of clusters designed for the data center. There are two key indicators to understand the network structure of the data center-the north-south data flow and the east-west data flow. The north-south data flow is the data flow from outside to inside the data center. The network structure may have routers, gateways, and there may be LoadBalance below, but it has nothing to do with the network associated with the container. We use the container way, similar to want to expand horizontally, based on the concept of this network, we want a more complex network structure.

Internet companies generally have less memory but more machines, often one or two thousand machines. However, some traditional enterprises use mainframes or machines that are relatively large. Their machines can form a network with four or five machines and have more memory. One machine can run two or three hundred containers, and four or five machines can run thousands of containers. This is the problem that Clico and some third-party solutions want to solve-- managing IP network segments, because the IP address is limited, and the real network segment is a set. And all of these sub-IP network segments are also real, to manage its conflicts. Now what you think of is to build an IP management interface, manage IP and fix it, but when there are more and more containers, there will be a lot of problems.

For the east-west solution of the data center, one of the methods proposed by Calico is based on BGP protocol. BGP protocol controls the routing rules, that is to say, in the case of no MAC address, the route has a routing table. When there are more routing tables, there are two things: first, broadcast, second, the routing table is large, and the software simulation will have performance bottlenecks and fine access control. Because the virtual network segments used between two network segments may conflict, there may be no problem with one or two network segments, and when dozens of APP are isolated, the two virtual network segments may use the same IP address. Make sure that the two virtual network segments are not connected, and that the two IP cannot conflict if they are connected. So is there a better way? Yes, MAC Vlan-- manages virtual machines as a host.

This only adds a layer of MAC address management to the Host model (instead of using your own virtual network, using the native host mode network). But it has a bigger problem is dependent on hardware, because MAC is to be transferred to IP, and IP address must be limited, MAC address is also limited, if there are conflicts are controlled by hardware, it can not be easily on the cloud, cloud MAC address is not controlled, basically on the basis of private cloud using MAC Vlan method to do. For Mesos solution we want to be universal, there is a network standard CNI, Calico is more flexible and guaranteed. This is the reason why such a scheme is recommended today.

# # the network is very complex. In the process of learning the network, there are several pieces of knowledge that are more important. First of all, what is a layer 3 network? It is actually layer 2 switching and layer 3 routing. The current router has been very advanced, one of the biggest features of Overlay is the use of routing as a basic knowledge point. You will see that after coming in from S0, there must be a routing table. After the routing table, some policy will be established, and then it will be hijacked on each network card to make a routing table and some rules. After arriving on the machine, this Brige has a new network card, which does not depend on the MAC address. Its advantage is that it can connect a very large network, containers can be set up at will, and hundreds of containers can be connected. All we have to do is match the net. But because the traditional Overlay is too powerful and advanced, it must be filled with countless headers in order to get from the farthest end to the inside. Vxlan to the latest representative of the SDN network is very advanced, but the software simulation of this network is very fragile, the larger the scale, the lower the efficiency, because the simplest problem is to unpack, unpack, and finally unpack the MAC address and IP. Network packets have the process of unpacking and unpacking, and only by making it have less ability to unpack and unpack can the rate be high. Generally speaking, all current solutions using Doker networks are lossy, unless they do not use or use Vxlan,Vxlan or virtual networks, and all virtual networks are lossy, so the current lossless way is impossible.

This packet header or routing table is very efficient when there are few rules and small scale. There are two lines of scarlet letters in the figure above. The first one is Overlay to Overlay. This is a pure Vxlan network. Swarm like Docker is created with this native network. Its problem is that it is virtual to convert, so the whole network is very weak. The loss can only reach 40% and 60%. This is not to say that the network is not advanced, the bottleneck is that it is a reference, if the hardware or third-party structure, can not be solved with Docker Swarm, then the network is basically unavailable. The second is Calico's Overlay, which has a loss of almost more than ten percent, which is the best way we have seen so far. This basic data comes from the company Percona, which makes a MySQL distribution that synchronizes between Master and Slave. The bottleneck of the increase in the amount of data can only be detected, so as to get this basic data report.

Doker is very easy to use, but Doker has built-in features of Swarm, and Doker Swarm is a network that can be created with two lines of command, which is very simple. This network is very suitable for debugging the network in the test environment and in stand-alone mode. It has its own DNS, name, network IP, and can also be managed by itself, which is very convenient. However, when it comes to the production level to use the network Mesos container solution, Calico this solution can help you better understand and solve this problem.

# # how is Calico done?

In the figure above, its principle is IP Table, using the components of Calico, using BGP to do, one of the key points is that it needs to save this routing rule and brush the route of each machine. It first needs to use etcd to save this information, because it cannot be stored locally, and it needs to be consistent. Second, its routing table needs to be brushed to other machines, and each machine has to pass it, so each machine is associated with etcd, triggering BGP to brush. It is centerless. After each machine is brushed, as soon as the routing table rules here change, the routing tables of all Mesos machines change, and it looks as if the network is connected. Because each machine comes out with the IP of S0, and when it gets there, it is also S0, but the route is transformed by rules, so both sides are connected, and the IP that can cross the host implements this process.

Here it underwent a small operation, because the more complex the network structure is, the more packets need to be done. When you jump in this network segment, you need to label the header of the message, and IP has to label it differently in order to get into that network. It makes a tradeoff, thinking that the network is not too much, it may be the size of a hundred machines, because there are ARP broadcasts, broadcast storms are very common in the network, and Mesos itself does not care about the network. What Calico does now is to add a Route Reflector in the middle. BIRD is also a third party and a storage of the Linux routing table. It uses the storage of a third party to store its own routing rules, and then ensures that the routing rules can be actively grasped by other parties, thus changing from the original way of pushing to the way of grasping, which simply solves this problem. It is with reference to this that Mesosphere implements a network scheme similar to Calico, called Minuteman, which is also open source. There is such a solution on open dcos, which you can refer to. This is the concrete implementation of Calico.

For the network structure of Mesos, we should not only have the concept of Framework, but also the concepts of Slave and Executer. Launch Calico makes a plug-in and installs it on each machine. When Framework calls and starts a task, it hijacks and triggers the enabling of IP rules. Calico has its own IP management, and it sends an isolation module to Slave to do policy isolation, get IP to do a Policy, and then do an isolation. This piece is to call IP Table, IP Set to do, and then update the Master information to put in the Zookeeper. Then I get an IP of Task. Especially in the absence of Doker, UnifiedContainerize is even more of this model. In fact, the installed component is a Calico BGP Agent, and a Felix,Agent brush local, Felix to brush someone else's.

Finally, the real IP network structure of Mesos is such a model: there is a small router on each machine. I define it as a router, which is actually a routing rule. After it is saved, the IP of the container in each machine can be different or the same, but the IP of the machine on the left is not the same as the IP of the machine on the right, and the routing table cannot be brushed. It can tell the one on the right by broadcasting, and then brush its own routing rules on the right. When visiting here, because its Package is cut with IP Table, it is similar to Router changing it, and destination is also changed and transmitted to another machine. There, because the routing rules have been transformed accordingly, the routing packet is added to it accordingly, and the container knows that the packet can be connected. Router side it is simulated, virtual, the more machines, because it has to take the initiative to push, its performance will be degraded. The way of Calico is to set up a central router on the outside, and then let the user swipe it, and then take it from the central router, thus reducing the number of broadcast nodes.

Mesosphere's latest approach is to use Erlang's model to build a P2P network structure. The advantage is that the route can be brushed point-to-point, that is, the connection between the applications of the two machines can be brushed only. If there is no connection, such as Calico will be brushed once, and every table will be brushed, because it does not know whether the user will start the new container in the next second, so this structure is the traditional network structure. But Mesosphere's Minuteman approach is a little more advanced, it is point-to-point to brush, when the container to brush, the limitation is that it can only be done in a data center, because it is an experimental project, we do not know its performance and scale.

The above is what the Mesos container network solution is, and the editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.