Shulou(Shulou.com)06/02 Report--

Flannel is an overlay network (Overlay Network) tool designed by the CoreOS team for Kubernetes, which is designed to help every CoreOS host using Kuberentes have a complete subnet.

Brief introduction

Flannel is a cross-host container network solution based on overlay network, that is, TCP packets are encapsulated in another network packet for routing, forwarding and communication. Flannel is developed by CoreOS and is a tool specially used for docker multi-computer interconnection, so that the containers created by hosts of different nodes in the cluster have unique virtual ip addresses in the cluster, and Flannel is written in gol.

Explanation of Flannel implementation principle

Flannel assigns a subnet to each host, and the container assigns IP from this subnet. These IP can be routed between host, and cross-host communication can be achieved without using nat and port mapping between containers.

Each subnet is divided from a larger IP pool, and flannel runs an agent called flanneld on each host, which is responsible for allocating subnet from the pool

Flannel uses etcd to store information such as network configuration, assigned subnet, IP of host, etc.

The forwarding of Flannel packets between hosts is implemented by backend, and currently supports a variety of backend, such as UDP, VxLAN, host-gw, AWS VPC and GCE routing.

Data forwarding proc

The container is accessed directly using the ip of the target container. By default, it is sent through the eth0 inside the container.

The message is sent to vethXXX via veth pair.

VethXXX is directly connected to the virtual switch docker0, and the message is sent through the virtual bridge docker0.

Looking up the routing table, the messages from the external container ip will be forwarded to the flannel0 virtual network card, which is a P2P virtual network card, and then the message will be forwarded to the flanneld listening on the other end.

Flanneld maintains the routing table between nodes through etcd, encapsulates the original message UDP and sends it out through the configured iface.

The message finds the target host through the network between hosts.

The message continues up to the transport layer and is processed by the flanneld program listening on port 8285.

The data is unpackaged and sent to the flannel0 virtual network card.

Look up the routing table and find that the message of the corresponding container should be delivered to docker0.

Docker0 finds the container connected to him and sends the message.

Flannel installation configuration environment prepares node name IP address installation software docker1 192.168.198.133 etcd, flannel, dockerdocker2 192.168.198.130 flannel, docker installation etcd

Etcd download address: https://github.com/coreos/etcd/releases

Start the command:

Etcd-name etcd1-data-dir / var/lib/etcd-- advertise-client-urls http://192.168.198.133:2379,http://127.0.0.1:2379-- listen-client-urls http://192.168.198.133:2379,http://127.0.0.1:2379 & install Flannel

Flannel download address: https://github.com/coreos/flannel/releases

Add flannel network configuration information to etcd:

Etcdctl-- endpoints http://127.0.0.1:2379 set / coreos.com/network/config'{"Network": "10.0.0.0 Network", "SubnetLen": 24, "SubnetMin": "10.0.1.0", "SubnetMax": "10.0.20.0", "Backend": {"Type": "vxlan"}}'

Network: used to specify a pool of Flannel addresses

SubnetLen: used to specify the length of the subnet mask of the ip segment of the docker0 assigned to a single host

SubnetMin: used to specify the minimum ip segment that can be allocated

SudbnetMax: used to specify the maximum ip segment that can be allocated. In the above example, each host can be assigned a subnet with a mask length of 24 bits. The subnet that can be assigned ranges from 10.0.1.0 to 10.0.20.0. This means that there can only be a maximum of 20 hosts in this segment.

Backend: used to specify how packets are forwarded. The default is udp mode. Host-gw mode has the best performance, but cannot cross host networks.

Start Flannel# cat / etc/systemd/system/flanneld.service [Unit] Description=FlanneldDocumentation= https://github.com/coreos/flannelAfter=network.targetBefore=docker.service[Service]User=rootExecStartPost=/usr/local/bin/mk-docker-opts.shExecStart=/usr/local/bin/flanneld\-- etcd-endpoints= "http://192.168.198.133:2379"\-- iface=192.168.198.133\-- ip-masq=true\-etcd-prefix=/coreos.com/networkRestart=on-failureType=notifyLimitNOFILE=65536 [Install] WantedBy=multi-user.target# systemctl daemon-reload# systemctl start flanneld

Resolution of Flannel startup process:

Get the configuration information of network from etcd, divide the subnet, register in etcd, and record the subnet information to / run/flannel/subnet.env. Flannel must be started before Docker.

Verify the Flannel network

View the data in etcd:

# etcdctl ls / coreos.com/network/subnets/coreos.com/network/subnets/10.0.18.0-24

View the flannel Nic information of docker1:

38: flannel.1:mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 1a:8e:8b:09:d6:d4 brd ff:ff:ff:ff:ff:ff inet 10.0.18.0/32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::188e:8bff:fe09:d6d4/64 scope link valid_lft forever preferred_lft forever

You can see that the address of the flannel0 network card is the same as the address stored by etcd, so the flannel network configuration is complete.

Configure Docker

After the Docker installation is complete, you need to modify its startup parameters so that it can use flannel for IP allocation and network communication

After Flannel runs, an environment variable file is generated, which contains the relevant parameters for the current host to communicate using flannel, as follows:

# cat / run/flannel/subnet.env FLANNEL_NETWORK=10.0.0.0/16FLANNEL_SUBNET=10.0.18.1/24FLANNEL_MTU=1450FLANNEL_IPMASQ=true

You can use the script provided by flannel to convert subnet.env into Docker startup parameters, and the created startup parameters are generated in the / run/docker_opts.env file by default:

# / opt/flannel/mk-docker-opts.sh-c # cat / run/docker_opts.envDOCKER_OPTS= "--bip=10.0.18.1/24-- ip-masq=false-- mtu=1450" modify the service startup file of docker as follows: # vim / lib/systemd/system/docker.serviceEnvironmentFile=/run/docker_opts.envExecStart=/usr/bin/dockerd $DOCKER_OPTS-H fd://

Restart docker

Systemctl daemon-reloadsystemctl restart docker

At this point, you can see that the ip of docker0 is already in the IP address range of the flannel network card:

3: docker0:mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:de:a3:d8:7d brd ff:ff:ff:ff:ff:ff inet 10.0.18.1/24 brd 10.0.18.255 scope global docker0 valid_lft forever preferred_lft forever inet6 fe80::42:deff:fea3:d87d/64 scope link valid_lft forever preferred_lft forever38: flannel.1:mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 1a: 8e:8b:09:d6:d4 brd ff:ff:ff:ff:ff:ff inet 10.0.18.0 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::188e:8bff:fe09:d6d4/64 scope link valid_lft forever preferred_lft forever 32 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::188e:8bff:fe09:d6d4/64 scope link valid_lft forever preferred_lft forever Verification Container Interconnection root@ubuntu:~# docker run-it busybox/ # ip A1: lo:mtu 65536 qdisc noqueue qlen 1000 link/loopback 00VO 00VO 00VO 00: 00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever39: eth0@if40:mtu 1450 qdisc noqueue link/ether 02:42:0a:00:12:02 brd ff:ff:ff:ff:ff:ff inet 10.0.18.2/24 brd 10.0.18.255 scope global eth0 valid_lft forever preferred_lft forever/ # ping 10.0.12.2PING 10 .0.12.2 (10.0.12.2): 56 data bytes64 bytes from 10.0.12.2: seq=0 ttl=62 time=0.803 msroot@ubuntuserver:~/flannel# docker exec-it b4d / bin/bashroot@b4d65fdc02a3:/usr/local/apache2# ip A1: lo:mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00vle00lv 0000lv 0000lv 00lv 00lv 00 brd 00lv 00lv 00lv 00lvvvvl 00lvv 00lv 00 inet 127.0.0.1pp 8 scope host lo Valid_lft forever preferred_lft forever5: eth0@if6:mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:00:0c:02 brd ff:ff:ff:ff:ff:ff inet 10.0.12.2/24 brd 10.0.12.255 scope global eth0 valid_lft forever preferred_lft foreverroot@b4d65fdc02a3:/usr/local/apache2# root@b4d65fdc02a3:/usr/local/apache2# root@b4d65fdc02a3:/usr/local/apache2# ping 10.0.18 .2ping 10.0.18.2 (10.0.18.2) 56 (84) bytes of data.64 bytes from 10.0.18.2: icmp_seq=1 ttl=62 time=0.794 ms64 bytes from 10.0.18.2: icmp_seq=2 ttl=62 time=0.957 ms

The flow of network packets at this time is shown in the figure:

Configure backend to host-gw

Host-gw bakcend is another backend of flannel. Unlike vxlan, host-gw does not encapsulate packets, but instead creates route entries in the host's routing table to the subnet of other hosts, enabling container network to communicate across hosts. It is important to note that host-gw cannot communicate across host networks, or physical routing support is required for cross-host network communications.

Modify the etcd as follows:

Etcdctl-- endpoints http://127.0.0.1:2379 set / coreos.com/network/config'{"Network": "10.0.0.0 Network", "SubnetLen": 24, "SubnetMin": "10.0.1.0", "SubnetMax": "10.0.20.0", "Backend": {"Type": "host-gw"}}'

Restart flanneld and docker:

Systemctl restart flanneld docker

Route entries can be viewed on the host machine:

Root@ubuntu:~# ip routedefault via 192.168.198.2 dev ens33 proto dhcp metric 100 10.0.12.0/24 via 192.168.198.130 dev ens33 10.0.18.0/24 dev docker0 proto kernel scope link src 10.0.18.1 linkdown 169.254.0.0/16 dev ens33 scope link metric 1000 192.168.198.0/24 dev ens33 proto kernel scope link src 192.168.198.133 metric 100

The original text is from: https://www.linuxprobe.com/flannel.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.