Shulou(Shulou.com)05/31 Report--

This article mainly introduces the example analysis of Kubernetes, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

One goal: container operation; two places and three centers; layer 4 service discovery; five kinds of Pod shared resources; six common CNI plug-ins; layer 7 load balancer; eight isolation dimensions; nine network model principles; ten types of IP addresses; hundred-level product lines; thousand-level physical machines; 10,000-level containers; like Wuyi, K8s has hundreds of millions of daily service people.

One goal: container operation

Kubernetes (k8s) is an open source platform for automating container operations. These container operations include deployment, scheduling, and expansion between node clusters.

Specific functions:

Automate container deployment and replication.

Real-time elastic shrinking container size.

Containers are grouped and load balancing between containers is provided.

Scheduling: on which machine the container runs.

Composition:

Kubectl: a client-side command line tool that acts as an operator for the entire system.

Kube-apiserver: provides an interface in the form of a REST API service as a control entry for the entire system.

Kube-controller-manager: perform the background tasks of the whole system, including the status of nodes, the number of Pod, the association of Pods and Service, etc.

Kube-scheduler: responsible for node resource management, receives tasks from kube-apiserver to create Pods, and assigns them to a node.

Etcd: responsible for service discovery and configuration sharing between nodes.

Kube-proxy: runs on each compute node and is responsible for the Pod network agent. Regularly obtain service information from etcd to make corresponding policies.

Kubelet: runs on each computing node, as an agent, receives the Pods tasks assigned to the node and manages the container, periodically obtains the container status, and feeds it back to kube-apiserver.

DNS: an optional DNS service for creating DNS records for each Service object so that all Pod can access the service through DNS.

The following is the architecture topology diagram of k8s:

Two places and three centers

The two places and three centers include the local production center, the local disaster preparedness center and the remote disaster preparedness center.

An important problem to be solved by the two places and three centers is the problem of data consistency.

K8s uses etcd components as a highly available, highly consistent service discovery repository. Used to configure sharing and service discovery.

It is a project inspired by Zookeeper and doozer. In addition to having all their functions, they also have the following four features:

Simple: HTTP+JSON-based API makes it easy to use the curl command.

Security: optional SSL customer authentication mechanism.

Fast: each instance supports a thousand writes per second.

Trustworthiness: the Raft algorithm is fully used to achieve distribution.

Four-tier service discovery

Let's take a picture to explain the layer 7 protocols of the network:

K8s provides two ways to make service discovery:

Environment variables: when you create a Pod, kubelet injects the relevant environment variables of all the Service in the cluster into the Pod. It is important to note that for an environment variable to inject a Service into a Pod, the Service must be created before the Pod. This almost makes this approach to service discovery unavailable.

For example, if a Service whose ServiceName is redis-master and the corresponding ClusterIP:Port is 10.0.0.11 ClusterIP:Port 6379, the corresponding environment variable is:

DNS: you can easily create a KubeDNS through cluster add-on to discover the services of the Service in the cluster.

The above two methods, one is based on TCP,DNS and based on UDP, they are based on the four-layer protocol.

Five kinds of Pod shared resources

Pod is the most basic operation unit of k8s and contains one or more closely related containers.

A Pod can be regarded as a "logical host" of the application layer by a containerized environment; multiple container applications in a Pod are usually tightly coupled, and the Pod is created, started, or destroyed on the Node; each Pod runs a special mount volume called Volume, so the communication and data exchange between them is more efficient. When designing, we can make full use of this feature to put a group of closely related service processes into the same Pod.

Containers in the same Pod can communicate with each other only through localhost.

An application container in a Pod shares five resources:

PID namespace: different applications in Pod can see the process ID of other applications.

Network namespaces: multiple containers in Pod can access the same IP and port range.

IPC namespace: multiple containers in Pod can communicate using SystemV IPC or POSIX message queues.

UTS namespace: multiple containers in Pod share a hostname.

Volumes (shared Storage Volume): individual containers in Pod can access the Volumes defined at the Pod level.

The life cycle of the Pod is managed by Replication Controller; it is defined by a template and then assigned to run on a Node, and when the container contained in the Pod ends, the Pod ends.

Kubernetes designed a unique set of network configurations for Pod, including assigning an IP address to each Pod and using the Pod name as the hostname for communication between containers. In the official account, the top architect replied "neat structure" to get a surprise gift package.

Six common plug-ins for CNI

CNI (Container Network Interface) container network interface is a set of standards and libraries for Linux container network configuration. Users need to develop their own container network plug-ins according to these standards and libraries. CNI only focuses on solving the problem of container network connection and resource release when the container is terminated, and provides a framework. So CNI can support a large number of different network models, and it is easy to implement.

The following diagram shows the six common CNI plug-ins:

Seven-tier load balancing

When it comes to load balancing, you have to mention the communication between servers first.

IDC (Internet Data Center) can also be called a data center or computer room, which is used to place servers. IDC network is the bridge of communication between servers.

There are a lot of network devices in the picture above. What are they for?

Routers, switches and MGW/NAT are all network devices, which are divided into different roles according to performance and internal and external networks.

Intranet access switch: also known as TOR (top of rack), is the server access network equipment. Each intranet access switch connects 40 to 48 servers and uses a network segment with a mask of / 24 as the server intranet segment.

Private network core switch: responsible for traffic forwarding and cross-IDC traffic forwarding of each intranet access switch within the IDC.

MGW/NAT:MGW, that is, LVS is used for load balancing, and NAT is used for address translation when private network devices access the public network.

External network core router: connect Meituan and unify the external network platform through static interconnection between operators or BGP.

First, let's talk about each layer of load balancer:

Layer 2 load balancing: layer 2 load balancing based on MAC address.

Layer 3 load balancing: load balancing based on IP address.

Layer 4 load balancing: load balancing based on IP+ port.

Layer 7 load balancer: load balancer based on application layer information such as URL.

Here is a diagram to show the difference between layer 4 and layer 7 load balancers:

The above four layers of service discovery are mainly about the native kube-proxy mode of K8s. K8s exposes services mainly through NodePort, binding a port on a minion host, and then forwarding Pod requests and load balancing. However, this method has the following shortcomings:

There may be many Service, if each bound a Node host port, the host needs to open the peripheral port for service invocation, management confusion.

The firewall rules required by many companies cannot be applied.

Ideally, through an external load balancer, bind a fixed port, such as 80, and forward it to the subsequent Service IP based on the domain name or service name.

Nginx addresses this requirement well, but the question is how to modify and load these Nginx configurations if new services are added.

The solution given by Kubernetes is Ingress. This is a scheme based on seven layers.

Eight isolation dimensions

K8s cluster scheduling needs to make a corresponding scheduling strategy for the isolation from top to bottom, from coarse-grained to fine-grained.

Nine principles of Network Model

The K8s network model should conform to four basic principles, three network requirements principles, one architecture principle and one IP principle.

Each Pod has a separate IP address, and it is assumed that all Pod are in a directly connected, flat cyberspace that can be accessed through Pod's IP, whether or not they are running on the same Node.

The IP of Pod in K8s is the smallest granularity IP. All containers within the same Pod share a network stack, which is called the IP-per-Pod model.

Pod the IP actually assigned by docker0.

The IP address and port seen inside the Pod are consistent with the outside.

Different containers in the same Pod share the network and can access each other's ports through localhost, similar to different processes within the same virtual machine.

IP-per-Pod model from the port allocation, domain name resolution, service discovery, load balancing, application configuration and other aspects, Pod can be regarded as an independent virtual machine or physical machine.

All containers can communicate with other containers without NAT.

All nodes can communicate with all containers in different NAT modes, and vice versa.

The address of the container is the same as the address seen by others.

To conform to the following architecture:

The concept of IP is derived from the architecture above, from outside the cluster to inside the cluster:

Ten types of IP addresses

It is well known that IP addresses are classified as ABCDE, and there are also five types of special-purpose IP.

The first category

Class A: 1.0.0.0-1226.255.255.255, the default subnet mask / 8, that is, 255.0.0.0.

Class B: 128.0.0.0-191.255.255.255, the default subnet mask / 16, that is, 255.255.0.0.

Class C: 192.0.0.0-223.255.255.255, the default subnet mask / 24, that is, 255.255.255.0.

Class D: 224.0.0.0-239.255.255.255, which is generally used for multicast.

Class E: 240.0.0.0-255.255.255.255 (where 255.255.255.255 is the network-wide broadcast address). Class E addresses are generally used for research purposes.

The second category

0.0.0.0

Strictly speaking, 0.0.0.0 is no longer a true IP address. It represents a collection of all unknown hosts and destination networks. The ambiguity here means that there is no specific entry in the local routing table indicating how to get there. As the default route.

127.0.0.1 local address.

The third category

224.0.0.1 Multicast address.

If your host turns on IRDP (internet route discovery, using multicast), then you should have such a route in your host routing table.

The fourth category

169.254.x.x

Use the DHCP function to automatically obtain the IP host, the DHCP server failure, or the response time is too long to exceed a system specified time, the system will assign you such an IP, which means that the network is not working properly.

The fifth category

10.xxx, 172.16.x.x~172.31.x.x, 192.168.x.x private address.

It is widely used within the enterprise. The purpose of keeping such an address is to avoid address confusion when accessing the public network.

Thank you for reading this article carefully. I hope the article "sample Analysis of Kubernetes" shared by the editor will be helpful to you. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.