Shulou(Shulou.com)05/31 Report--

This article mainly shows you "how to configure VXLAN network on Linux", the content is easy to understand, clear, hope to help you solve your doubts, let the editor lead you to study and learn "how to configure VXLAN network on Linux" this article.

1. Peer to peer VXLAN

Let's take a look at the simplest peer-to-peer VXLAN network. Peer-to-peer VXLAN is a VXLAN network built by two hosts, and each host has a VTEP,VTEP that communicates through their IP address. The point-to-point VXLAN network topology diagram is shown in the figure:

In order not to affect the network environment of the host, we can use Linux VRF to isolate the routing of root network namespace. VRF (Virtual Routing and Forwarding) is a routing instance composed of a routing table and a group of network devices, which you can understand as lightweight network namespace, which only virtualizes three layers of network protocol stack, while network namespace virtualizes the whole network protocol stack. For more information, see the principle and implementation of Linux VRF (Virtual Routing Forwarding).

VRF is not supported until the Linux Kernel version is greater than 4.3.It is recommended that the students doing this experiment upgrade the kernel first.

Of course, if you have a clean host specifically for experiments, you can isolate it without VRF.

Let's combine VRF to create a peer-to-peer VXLAN network.

First create the VXLAN interface on 192.168.57.50:

$ip link add vxlan0 type vxlan\ id 42\ dstport 4789\ remote 192.168.57.54\ local 192.168.57.50\ dev eth0

Explanation of important parameters:

Id 42: specifies the value of VNI. Valid values are between 1 and $2 ^ {24} $.

Dstport: the port for VTEP communication, and the port assigned by IANA is 4789. If not specified, Linux defaults to 8472.

Remote: the address of the peer VTEP.

Local: the IP address to be used by the current node VTEP, that is, the IP address of the current node tunnel port.

Dev eth0: the device used by the current node for VTEP communication to obtain the VTEP IP address. This parameter has the same purpose as the local parameter, just choose one of the two.

View the details of vxlan0:

$ip-d link show vxlan011: vxlan0: mtu 1500 qdisc noqueue master vrf-test state UNKNOWN mode DEFAULT group default qlen 1000 link/ether 82:f3:76:95:ab:e1 brd ff:ff:ff:ff:ff:ff promiscuity 0 vxlan id 42 remote 192.168.57.54 local 192.168.57.50 srcport 00 dstport 4789 ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx

Next, create a VRF and bind the vxlan0 to the VRF:

$ip link add vrf0 type vrf table 10$ ip link set vrf0 up$ ip link set vxlan0 master vrf0

Check vxlan0's information again:

Ip-d link show vxlan013: vxlan0: mtu 1500 qdisc noqueue master vrf0 state UNKNOWN mode DEFAULT group default qlen 1000 link/ether aa:4d:80:e3:75:e0 brd ff:ff:ff:ff:ff:ff promiscuity 0 vxlan id 42 remote 192.168.57.54 local 192.168.57.50 srcport 00 dstport 4789 ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx vrf_slave table 10 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535

You will find more information about VRF.

Next, configure the IP address for vxlan0 and enable it:

$ip addr add 172.18.1.2 ip addr add 24 dev vxlan0 $ip link set vxlan0 up

After successful execution, you will find that the following is added to the VRF routing table entry, and all network packets with the destination address of 172.18.1.0mm24 are to be forwarded through vxlan0:

$ip route show vrf vrf0172.18.1.0/24 dev vxlan0 proto kernel scope link src 172.18.1.2

A FDB forwarding table will also be added:

$bridge fdb show00:00:00:00:00:00 dev vxlan0 dst 192.168.57.54 self permanent

This entry means that the default VTEP peer address is 192.168.57.54. In other words, after the original message passes through vxlan0, the VXLAN header is added by the kernel, and the destination IP address of the external UDP header is labeled 192.168.57.54.

Do the same configuration on another host (192.168.57.54):

$ip link add vxlan0 type vxlan id 42 dstport 4789 remote 192.168.57.50$ ip link add vrf0 type vrf table 10$ ip link set vrf0 up$ ip link set vxlan0 master vrf0 $ip addr add 172.18.1.3 ip link set vxlan0 up 24 dev vxlan0 $ip link set vxlan0 up

When everything is done, you can communicate with each other, ping 172.18.1.3 on 192.168.57.50:

$ping 172.18.1.3-I vrf0

At the same time, use wireshark to grab packets remotely:

$ssh root@192.168.57.54 'tcpdump-I any-s0-c 10-nn-w-port 4789' | / Applications/Wireshark.app/Contents/MacOS/Wireshark-k-I-

I will not explain the specific meaning, refer to the Tcpdump sample tutorial.

You can see that the VXLAN message can be divided into three pieces:

The innermost layer is the messages seen by the actual communicating entities in the overlay network (such as the ARP request here), which are no different from the communication messages in the classical network, except that some messages are relatively small because of MTU.

The middle layer is the VXLAN header, and our most concerned field, VNI, is indeed 42.

The outermost layer is the communication header of the host where the VTEP is located, and the destination IP address is the peer 192.168.57.54.

Let's analyze the process of vxlan communication in this simplest mode:

Send the ping message to 172.18.1.3, look at the routing table, and the message will be sent from the vxlan0.

The kernel finds that the IP of vxlan0 is 172.18.1.2 IP 24, which is on the same network segment as the destination IP, so in the same local area network, you need to know the MAC address of the other party, so you will send an ARP message query.

The source MAC address of the ARP message is the MAC address of vxlan0, and the destination MAC address is the all-1 broadcast address (ff:ff:ff:ff:ff:ff).

VXLAN adds the header according to the configuration (VNI 42).

The VTEP address of the peer is 192.168.57.54, to which the message is sent.

When the peer host receives this message, the kernel finds that it is a VXLAN message and will send it to the corresponding VTEP according to VNI.

VTEP removes the VXLAN header and takes out the real ARP request message. At the same time, VTEP records the source MAC address and IP address information to the FDB table, which is a learning process. Then an ARP reply message is generated.

$bridge fdb show 00vvl 00lv 0000vl 00lv 00 dev vxlan0 dst 192.168.57.50 self permanentaa:4d:80:e3:75:e0 dev vxlan0 dst 192.168.57.50 self

The MAC address of the reply message destination is the MAC address of the sender VTEP, and the destination IP is the IP address of the sender VTEP, which is sent directly to the destination VTEP.

The reply message is returned directly to the sender host through the underlay network, and the sender host forwards the message to VTEP,VTEP to unpack the ARP reply message according to the ARP, adds the ARP cache to the kernel, and learns the IP address and the destination MAC address of the destination VTEP according to the message, and adds it to the FDB table.

$ip neigh show vrf vrf0 172.18.1.3 dev vxlan0 lladdr 76:06:5c:15:d9:78 STALE $bridge fdb show 00VO 00VO 00VO 00 dev vxlan0 dst 192.168.57.54 self permanentfe:4a:7e:a2:b5:5d dev vxlan0 dst 192.168.57.54 self

At this point, VTEP already knows all the information needed for communication, and the subsequent ping messages of ICMP are unicast in this logical tunnel, so it is no longer necessary to send ARP message queries.

To sum up the above process: the ping message of a VXLAN network goes through two processes: ARP addressing and ICMP response. Once the VTEP device learns the other party's ARP address, the subsequent communication can avoid the process of ARP addressing.

2. VXLAN + Bridge

In the above point-to-point VXLAN network communication, there is only one VTEP and only one communication entity, but in actual production, there are dozens or even hundreds of virtual machines or containers on each host that need to communicate, so a mechanism is needed to organize these communication entities and forward them through the tunnel entrance VTEP.

The solution is also very common. Linux Bridge can connect multiple virtual network cards together, so you can choose to use Bridge to put multiple virtual machines or containers on the same VXLAN network. The network topology diagram is shown in the figure:

Compared with the above mode, there is only one more Bridge to connect the veth pair in different network namespace, and the VXLAN Nic also needs to connect to this Bridge.

First create the VXLAN interface on 192.168.57.50:

$ip link add vxlan0 type vxlan\ id 42\ dstport 4789\ local 192.168.57.50\ remote 192.168.57.54

Then create a bridge bridge0, bind the VXLAN Nic vxlan0 to it, then bind the bridge0 to the VRF, and start them:

$ip link add br0 type bridge$ ip link set vxlan0 master br0 $ip link add vrf0 type vrf table 10$ ip link set br0 master vrf0 $ip link set vxlan0 up$ ip link set br0 up$ ip link set vrf0 up

Here, create network namespace and a pair of veth pair, bind one end of veth pair to the bridge, then put the other end to network namespace and bind the IP address 172.18.1.2:

$ip netns add ns0 $ip link add veth0 type veth peer name eth0 netns ns0 $ip link set veth0 master br0 $ip link set veth0 up$ ip-n ns0 link set lo up$ ip-n ns0 addr add 172.18.1.2 ip 24 dev eth0 $ip-n ns0 link set eth0 up

Configure the VXLAN network on another host in the same way, binding 172.18.1.3 to eth0 in another network namespace:

$ip link add vxlan0 type vxlan\ id 42\ dstport 4789\ local 192.168.57.54\ remote 192.168.57.50$ ip link add br0 type bridge$ ip link set vxlan0 master br0 $ip link add vrf0 type vrf table 10$ ip link set br0 master vrf0 $ip link set vxlan0 up$ ip link set br0 up$ ip link set vrf0 up$ ip netns add ns0 $ip link add veth0 type veth peer name eth0 netns ns0 $ip link set veth0 master br0 $ip link set veth0 up$ ip-n ns0 link set lo up$ ip-n ns0 addr add 172.18 .1.3 take 24 dev eth0 $ip-n ns0 link set eth0 up

From 172.18.1.2 ping 172.18.1.3, it is found that the whole communication process is similar to the previous experiment, except that the ARP message sent by the container will first pass through the bridge and then be forwarded to vxlan0, and then the VXLAN header will be added by the Linux kernel at vxlan0, and finally sent to the peer.

Logically, the network cards in the network namespace on different hosts in the VXLAN network are connected to the same bridge, so that multiple containers under the same VXLAN network can be created on the same host and communicate with each other.

3. VXLAN in Multicast Mode

The above two modes can only connect point to point, that is to say, there can only be two nodes in the same VXLAN network. Is there a way to accommodate multiple nodes in the same VXLAN network? Let's first review two key messages of VXLAN communications:

MAC address of the virtual machine (or container) of the other party

The IP address of the host where the other party is located (that is, the IP address of the peer VTEP)

Containers across hosts need to know each other's MAC address when communicating for the first time, so an ARP message query is sent. If there are multiple nodes, the ARP query message will be sent to all nodes, but the traditional ARP message broadcast can not be done, because Underlay and Overlay are not in the same layer 2 network, ARP broadcast can not escape the host by default. In order to realize the broadcast of the Overlay network, the message must be sent to all the nodes where the VTEP resides. In order to solve this problem, there are probably two ways of thinking:

Using multicast, some nodes in the network are formed into a virtual whole.

Know the MAC address and VTEP IP information in advance, and directly tell the sender VTEP the ARP and FDB information. Generally, this information is collected through an external distributed control center, and the collected information is distributed to all nodes in the same VXLAN network.

Let's first take a look at how multicast is implemented, and the distributed control center will save it for the next article.

If VXLAN is to use multicast mode, the underlying network needs to support multicast, with a multicast address range of 224.0.0.02239.255.255.255.

Compared with the peer-to-peer VXLAN + Bridge mode above, the peer parameter is changed to group parameter, and the other parameters remain unchanged. The command is as follows:

# execute $ip link add vxlan0 type vxlan\ id 42\ dstport 4789\ local 192.168.57.50\ group 224.1.1.1$ ip link add br0 type bridge$ ip link set vxlan0 master br0 $ip link add vrf0 type vrf table 10$ ip link set br0 master vrf0 $ip link set vxlan0 up$ ip link set br0 up$ ip link set vrf0 up$ ip netns add ns0 $ip link add veth0 type veth peer name eth0 netns ns0 $ip link set veth0 master br0 $ip link set veth0 up$ ip on host 192.168.57.50 N ns0 link set lo up$ ip-n ns0 addr add 172.18.1.2 ip link add br0 type bridge$ ip link set vxlan0 master br0 24 dev eth0 $ip-n ns0 link set eth0 up# execute $ip link add vxlan0 type vxlan\ id 42\ dstport 4789\ local 192.168.57.54\ group 224.1.1.1$ ip link add br0 type bridge$ ip link set vxlan0 master br0 $ip link add vrf0 type vrf table 10$ ip link set br0 master vrf0 $ip link set vxlan0 up$ ip link set br0 up$ ip link on host 192.168.57.54 Set vrf0 up$ ip netns add ns0 $ip link add veth0 type veth peer name eth0 netns ns0 $ip link set veth0 master br0 $ip link set veth0 up$ ip-n ns0 link set lo up$ ip-n ns0 addr add 172.18.1.3 ip 24 dev eth0 $ip-n ns0 link set eth0 up

What is obviously different from the above experiment is the content of the FDB entry:

$bridge fdb show00:00:00:00:00:00 dev vxlan0 dst 224.1.1.1 self permanent

The value of the dst field becomes the multicast address 224.1.1.1 instead of the previous VTEP address of the other party. VTEP will join the same multicast group 224.1.1.1 through IGMP (Internet Group Management Protocol).

Let's analyze the whole process of VXLAN communication in multicast mode:

Send the ping message to 172.18.1.3, look at the routing table, and the message will be sent from the vxlan0.

The kernel finds that the IP of vxlan0 is 172.18.1.2 IP 24, which is on the same network segment as the destination IP, so in the same local area network, you need to know the MAC address of the other party, so you will send an ARP message query.

The source MAC address of the ARP message is the MAC address of vxlan0, and the destination MAC address is the all-1 broadcast address (ff:ff:ff:ff:ff:ff).

VXLAN adds the header according to the configuration (VNI 42).

At this stage, it is not the same as before. Because you do not know which host the peer VTEP is on, according to the multicast configuration, VTEP will send a multicast message to the multicast address 224.1.1.1.

All hosts in the multicast group will receive this message. When the kernel finds that it is a VXLAN message, it will send it to the corresponding VTEP according to the VNI.

The VTEP of all hosts that receive the message will remove the VXLAN header and take out the real ARP request message. At the same time, VTEP records the source MAC address and IP address information to the FDB table, which is a learning process. If you find that the ARP is not sent to yourself, it will be discarded directly; if it is sent to yourself, an ARP reply message will be generated.

The following steps are the same as the above experiment.

The whole communication process is similar to that before, except that Underlay uses multicast to send messages, which is relatively simple and efficient for multi-node VXLAN networks. However, multicast also has its problems. Not all network devices support multicast (such as public cloud). Coupled with the packet waste caused by multicast, it is rarely used in actual generation.

The above is all the contents of the article "how to configure VXLAN Network on Linux". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.