Shulou(Shulou.com)06/01 Report--

This article mainly introduces "introduction to the principle of VXLAN protocol". In daily operation, I believe that many people have doubts about the introduction of the principle of VXLAN protocol. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "introduction to the principle of VXLAN protocol"! Next, please follow the editor to study!

VXLAN (Virtual eXtensible Local Area Network, Virtual Extensible Local area Network) is a virtual tunnel communication technology. It is an Overlay (overlay network) technology, through the three-layer network to build a virtual two-layer network.

To put it simply, VXLAN uses tunneling technology on the underlying physical network (underlay) to decouple the logical network from the physical network and realize flexible networking requirements with the help of the Overlay logical network constructed by the UDP layer. It has almost no impact on the original network architecture, and a new layer of network can be set up without any changes to the original network. It is also because of this feature that many CNI plug-ins (container network interfaces in Kubernetes clusters, which we all know, if you don't know, now you know) choose VXLAN as the communication network.

VXLAN supports not only one-to-one, but also one-to-many. A VXLAN device can learn other peer-to-peer IP addresses like a bridge, and can also configure static forwarding tables directly.

A typical data center VXLAN network topology diagram is shown in the figure:

VM refers to the virtual machine and Hypervisor refers to the virtualization manager.

1. Why do I need VXLAN?

Compared with VLAN, VXLAN is obviously much more complex, coupled with the first-mover advantage of VLAN, has been widely supported, so why do you need VXLAN?

VLAN ID quantity limit

VLAN tag has a total of 4 bytes, of which 12 bit is used to identify different layer 2 networks (that is, LAN ID), so it can only support a maximum of $2 ^ {12} $, that is, 4096 subnets. With the rise of virtualization (virtual machines and containers), there will be thousands of machines in a data center that need to communicate, and VLAN will not be able to meet the demand. The VXLAN message Header reserves 24 bit to identify different layer 2 networks (VNI,VXLAN Network Identifier), that is, 3 bytes, which can support $2 ^ {24} $subnets.

Switch MAC address table limit

For the communication with the host of the same network segment, after the switch, the message will be queried by the MAC address table for layer 2 forwarding. After the data center is virtualized, the number of VM increases by an order of magnitude compared with the original physical machine, and after the application containerization, the container also increases by an order of magnitude compared with VM.. The memory of the switch is limited, so the MAC address table is also limited. With the unprecedented increase in the number of virtual machine (or container) network card MAC addresses, the switch is under great pressure.

VXLAN, on the other hand, is amazing. It encapsulates layer 2 Ethernet frames in UDP using VTEP (explained later). A VTEP can be shared by all VM (or containers) on a physical machine, and a physical machine corresponds to a VTEP. From the point of view of the switch, only the UDP data is being transferred between different VTEP. You only need to record the MAC address table entries equal to the number of physical machines, and everything is back to the same as before.

The migration range of virtual machine or container is limited

With the integration of VLAN and physical network, there is no Overlay network, which brings the problem that virtual network can not break the limitation of physical network. For example, if a virtual machine (or container) is to be deployed on VLAN 100, it can only be deployed on a physical device that supports VLAN 100.

In fact, VLAN also has a solution, which is to connect all the switch Trunk to create a large layer 2, which brings the problem of excessive expansion of the broadcast domain, including more unknown unicast and multicast, that is, BUM (Broadcast,Unknown Unicast,Multicast). At the same time, the MAC address table of the switch will also have unbearable problems.

VXLAN encapsulates layer 2 Ethernet frames in UDP (mentioned above), which is equivalent to building a layer 2 network on layer 3 network. In this way, no matter whether your physical network is layer 2 or layer 3, it does not affect the network communication of virtual machines (or containers), it does not matter which physical device is deployed, and can be migrated at will.

Generally speaking, the traditional layer 2 and layer 3 networks are unable to cope with these requirements. Although many improved technologies such as stacking, SVF, TRILL and so on can increase the scope of layer 2 and strive to improve the classical network, it is very difficult to make as few changes to the network as possible while ensuring flexibility. In order to solve these problems, many solutions have been put forward, Overlay is one of them, and VXLAN is a typical technical solution of Overlay. Here is a brief introduction to Overlay.

2. What is Overlay?

In the field of network technology, Overlay refers to a virtual technology model superimposed on the network architecture. Its general framework is to realize the bearing of applications on the network without large-scale modification of the basic network, and can be separated from other network services, and is mainly based on the basic network technology based on IP.

IETF puts forward three major technical solutions: VXLAN, NVGRE and STT in the field of Overlay technology. The general idea is to carry Ethernet packets to a certain tunnel level, the difference lies in the selection and construction of the tunnel, while the bottom layer is IP forwarding. VXLAN and STT have lower requirements for traffic balancing for existing network devices, that is, load link load sharing has good adaptability. General network devices can perform link aggregation or equal cost routing traffic balancing for L2-L4 data content parameters, while NVGRE requires network equipment to be aware of GRE extension headers and HASH flow ID, and hardware upgrades are required. STT has made great changes to TCP, the tunnel mode is close to UDP, and the tunnel construction technology is innovative and complex, while VXLAN makes use of the existing general UDP transmission with high maturity.

Overall, VLXAN technology has greater advantages, and the current VLXAN has been supported by more manufacturers and customers, and has become the mainstream standard of Overlay technology.

3. Principle of VXLAN protocol

VXLAN has several common terms:

VTEP (VXLAN Tunnel Endpoints,VXLAN Tunnel Endpoint)

The edge device of VXLAN network, which is used to process VXLAN messages (packet and unpack). The VTEP can be either a network device (such as a switch) or a machine (such as a host in a virtualized cluster).

VNI (VXLAN Network Identifier,VXLAN Network Identifier)

VNI is the identity of each VXLAN segment and is a 24-bit integer with a total of $2 ^ {24} = 16777216 $(more than 10 million). Generally, each VNI corresponds to one tenant, which means that a public cloud built using VXLAN can theoretically support 10 million tenants.

Tunnel (VXLAN Tunnel)

Tunneling is a logical concept, and there is no specific physical entity correspondence in the VXLAN model. Tunnel can be regarded as a virtual channel. Both sides of VXLAN communication think that they are communicating directly and do not know the existence of the underlying network. On the whole, each VXLAN network seems to build a separate communication channel, that is, a tunnel, for the communication virtual machine.

The figure above shows the working model of VXLAN, which is created on the original IP network (layer 3) and can deploy VXLAN as long as it is reachable at layer 3 (capable of communicating with each other through IP). There is a VTEP device at each endpoint of the VXLAN network, which is responsible for unpacking and encapsulating the VXLAN protocol message, that is, encapsulating the header of the VTEP communication on the virtual message.

Multiple VXLAN networks can be created on the physical network, and these VXLAN networks can be regarded as a tunnel, and virtual machines / containers on different nodes can be directly connected through the tunnel. Different VXLAN networks are identified by VNI, so that different VXLAN can be isolated from each other.

The message structure of VXLAN is shown in the following figure:

VXLAN Header: add an 8-byte VXLAN header in front of the original layer 2 frame, the most important of which is VNID, which takes up 3 bytes (that is, 24 bit), similar to VLAN ID, and can have $2 ^ {24} $network segments.

UDP Header: 8-byte UDP header is used in front of the VXLAN and the original layer 2 frame for encapsulation (MAC IN UDP). The destination port number defaults to 4789, and the source port is randomly assigned by stream (through MAC,IP, layer 4 port number for hash operation), so that ECMP can be better done.

IANA (Internet As-signed Numbers Autority) assigns 4789 as the default destination port number for VXLAN.

After the layer 2 encapsulation added above, add the IP header (20 bytes) and MAC header (14 bytes) of the underlying network, where IP and MAC are the IP address and MAC address of the host.

At the same time, we need to pay attention to the problem of MTU. The traditional network MTU is generally 1500. Here, the extra encapsulation of VXLAN is added (36mm 14amp 18, in the case of 14 is access port, 4 bytes of VLAN Tag is omitted) 50 or 54 bytes, and the MTU needs to be adjusted to 1550 or 1554 to prevent frequent subpackets.

Flood and Learn of VXLAN

Generally speaking, the forwarding process of VXLAN message is as follows: the original message goes through VTEP, the VXLAN header and the outer UDP header are added by the Linux kernel, and then sent out. After receiving the VXLAN message, the peer VTEP removes the outer UDP header, and sends the original message to the destination server according to the VNI of the VXLAN header. But here is a question: how do the two sides know all the communication information before the first communication? This information includes:

Which VTEP needs to be added to the same VNI group?

How does the sender know the MAC address of the other party?

How do I know which node the destination server is on (that is, the address of the destination VTEP)?

The first question is simple. VTEP is usually configured by a network administrator. To answer the last two questions, you have to go back to the VXLAN protocol message to see what information a complete VXLAN message needs:

Inner message: the IP addresses of both sides of the communication are clear, and only VXLAN is needed to fill each other's MAC address, so a mechanism is needed to implement the ARP function.

VXLAN header: all you need to know is VNI. Generally, it is directly configured on the VTEP, either planned in advance or automatically generated according to the inner message.

UDP header: you need to know the source port and destination port. The source port is automatically generated by the system, and the destination port is 4789 by default.

IP header: you need to know the IP address of the peer VTEP, which is the most important part.

In fact, VTEP also has its own forwarding table, which is maintained by flooding and learning mechanisms. For unknown unicast and broadcast traffic where the destination MAC address does not exist in the forwarding table, it will be flooded to all VTEP except the source VTEP. After the destination VTEP responds to the packet, the source VTEP will learn the mapping relationship between MAC,VNI and VTEP from the packet and add it to the forwarding table. Later, when the packet is forwarded to this MAC address VTEP gets the destination VTEP address directly from the forwarding table and sends unicast data to the destination VTEP.

VTEP forwarding tables can be learned in two ways:

Multicasting

External control center (CNI plug-ins such as Flannel, Cilium, etc.)

MAC header: determine the IP address of the VTEP, and then it will be easy. The MAC address can be obtained through the classic ARP method.

4. VXLAN of Linux

Linux's support for the VXLAN protocol was not long before Stephen Hemminger incorporated the related work into kernel in 2012, and eventually appeared in kernel version 3.7.0. For the sake of stability and many features, you may see that some software recommends using VXLAN on kernel versions 3.9.0 or later.

By kernel 3.12, Linux has complete support for VXLAN, unicast and multicast, IPv4 and IPv6. Using man to view the link subcommand of ip, you can see if there is a VXLAN type:

$man ip-link

Search VXLAN and you can see the following description:

Manage VXLAN Interfac

The basic management of Linux VXLAN API is as follows:

Create a point-to-point VXLAN interface:

$ip link add vxlan0 type vxlan id 4100 remote 192.168.1.101 local 192.168.1.100 dstport 4789 dev eth0

Where the id is VNI,remote, the IP,local is the remote host, and the IP,dev of your local host represents the interface from which the VXLAN data is transmitted.

In VXLAN, the VXLAN interface (in this case, vxlan0) is generally referred to as VTEP.

Create a VXLAN interface for multicast mode:

$ip link add vxlan0 type vxlan id 4100 group 224.1.1.1 dstport 4789 dev eth0

Multicast groups mainly learn MAC addresses through ARP flooding, that is, broadcast ARP requests in the VXLAN subnet, and then respond to the corresponding nodes. Group specifies the address of the multicast group.

View the details of the VXLAN interface:

$ip-d link show vxlan0

FDB table

FDB (Forwarding Database entry, that is, forwarding table) is a layer 2 forwarding table maintained by the Linux bridge, which is used to store the MAC address of the remote virtual machine / container, the remote VTEP IP, and the mapping of the VNI. You can manipulate the FDB table through the bridge fdb command:

Entry add:

$bridge fdb add dev dst

Entry deletion:

$bridge fdb del dev

Entry update:

$bridge fdb replace dev dst

Entry query:

$bridge fdb show

At this point, the study of "introduction to the principles of VXLAN protocol" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.