Shulou(Shulou.com)06/02 Report--

I. Background

Openstack is adopted by the majority of public cloud vendors. For public cloud scenarios, the network functions provided by Newtron components have always been difficult to understand. This article describes in detail the north-south and east-west traffic implementation of vm under the openstack centralized network node architecture.

II. Functions of network nodes

Because in the default deployment mode of openstack, compute nodes implement Layer 2 interworking through the ml2 plug-in, all Layer 3 traffic must pass through network nodes, as shown in the following figure:

In the figure, there are two different subnets under the same tenant, vm1/2 and vm3/4 belong to different subnets respectively. From the figure above, it can be seen that communication between different subnets, communication between VMs without fip and public network, and access to VMs with fip through public network need to pass through network nodes. There is a single point hidden danger in network nodes, so it is necessary to make high availability of this node through L3HA.

Third, the network type in openstack

There are two types of network modes in openstack: provider networks and self-service networks. Provider is a semi-virtualized Layer 2 network architecture, which can only be implemented by bridging. In the provider network mode, the IP address obtained by vm is on the same network segment as the physical network. It can be regarded as an extension of the physical network. In this mode, the control node does not need to install L3 agents or network nodes. The vm communicates directly with the physical network through the NIC of the host. The provider network only supports flat and VLAN modes. Its structure is as follows:

The self-service mode allows tenants to create their own networks, and finally the network created by tenants accesses the external network by NAT with the help of the provider network. Therefore, the self-service mode can be regarded as an extension of the network hierarchy. To implement the self-service mode, a provider network must be created first. The self-service network supports flat, vlan, vxlan, and gre modes. Its structure is as follows:

The IP address obtained by vm from self-service is called fix IP, and the IP address obtained by vm from provider network is called floating IP. Regardless of whether the network type created by the tenant is vxlan or vlan (br-tun or br-vlan), the traffic between tenant VMs via fix ip is called east-west direction. Only when vm needs to access the external network via snat or vm via fip, the traffic at this time is called north-south direction.

IV. Internal components of network nodes

Compared with computer nodes, network nodes have dhcp and router spaces for tenants and br-ex for connecting to the internet (compute nodes also have OVS for br-int and br-tun/vlan, see other blog posts by the author).

The internal component communication of a network node is shown in the following figure:

From the above figure, we know that fix ip finally needs to use the iptables rule in the router namespace to realize vm access to the external network or be accessed by the external network in the way of sant or dnat. Of course, VMs can also be directly connected to the provider network, but the provider network is only a layer 2 virtualized network, and physical equipment is needed to achieve layer 3 and above functions. Take the self-service mode as an example to explain the east-west and north-south traffic trends of vm in detail.

V. East-west traffic analysis of vm

As we know above, a tenant can create multiple subnets. In this case, the communication between VMs is divided into two cases: the same subnet and different subnets. The difference is that the communication between different subnets needs to pass through the gateway (network node), while the communication between the same subnet does not need to pass through the network node.

1. Communication between different subnets

Regardless of whether the network type created by the tenant is tunnel or vlan, communication between different subnets must be completed by means of L3 Agent, while in the centralized network node architecture, only the network node deploys this role, and the traffic between VMs is shown in the following figure:

1. vm1 sends a communication request to vm2, and according to the destination IP address, it knows that vm2 and itself are not in the same network segment, and sends the data packet to the gateway.

2. Packets are checked by iptables security policy on linux bridge, pressed and sent to br-int with internal vlan.

3. The packet takes off the internal vlan number of br-int and enters br-tun/br-vlan.

4. Packets entering br-tun are encapsulated by vxlan and marked with vni number at this time, packets entering br-vlan are marked with external vlan number at this time, and leave compute1 through nic.

5. The data packet enters the network node, and the data packet passing through br-tun completes the decapsulation of vxlan to remove the vni number, and the data packet passing through br-vlan removes the vlan number.

6. Packets enter br-int, where they are marked with an internal vlan.

7. Enter the router namespace and find the gateway. The gateway of VM1 is configured on port qr-1 of this router.

8. Route the packet to VM2 gateway, VM2 gateway is configured on QR-2 port, send it back to BR-INT and type the internal VLAN number.

9. The packet takes off the internal vlan number of br-int and enters br-tun/br-vlan.

10. Packets entering br-tun are encapsulated by vxlan and marked with vni number at this time, and packets entering br-vlan are marked with external vlan number at this time, and leave network node through nic.

11. The data packet enters Compute2, and the VXLAN decapsulation is completed for the data packet passing through br-tun to remove the VNi number, and the VLAN number is removed for the data packet passing through br-vlan.

12. Packets enter br-int, where they are marked with an internal vlan.

13. Packets leave br-int and remove the internal vlan, and are sent to linux bridge to be checked for iptables security policies.

14. The final packet is sent to VM2.

2. Communication between identical subnets

Communication between the same subnet does not need to rely on L3 Agent, and the traffic between vm is shown in the following figure:

vm1 initiates a request to vm2, and knows that vm2 is on the same network segment as itself through the destination IP address.

2. The data packet passes through linux bridge, performs security policy check, and enters br-int to be marked with internal vlan.

3. The packet takes off the internal vlan number of br-int and enters br-tun/br-vlan.

4. Packets entering br-tun are encapsulated by vxlan and marked with vni number at this time, packets entering br-vlan are marked with external vlan number at this time, and leave compute1 through nic.

5. The data packet enters Compute2, and the VXLAN decapsulation is completed for the data packet passing through br-tun to remove the VNi number, and the VLAN number is removed for the data packet passing through br-vlan.

6. Packets enter br-int, where they are marked with an internal vlan.

7. Packets leave br-int and remove the internal vlan, and are sent to linux bridge to be checked for iptables security policies.

8. The final packet is sent to VM2.

VI. North-South flow analysis of vm

North-south traffic is also divided into floating ip and non-floating ip (fix ip) two cases, the only difference is that vm finally leaves the network node to access the internet, vm floating ip source address is floating ip, and use fix ip vm through snat mode, source address is network node ip, vm north-south traffic as shown in the following figure:

vm1 sends a communication request to the public network, and the data packet is sent to the gateway.

2. Packets are checked by linux bridge through iptables security policies, pressed and sent to br-int with an internal vlan.

3. The data packet takes off the internal vlan number of br-int and enters br-tun/br-vlan. The data packet entering br-tun is encapsulated by vxlan and marked with vni number at this time. The data packet entering br-vlan is marked with external vlan number at this time and leaves compute1 through nic.

4. The data packet enters the network node. The data packet passing through br-tun completes the decapsulation of vxlan and removes the vni number. The data packet passing through br-vlan removes the vlan number and is sent to br-int. At this time, the internal vlan number will be marked.

5. Enter the router namespace to query the routing table entries. The gateway of VM1 is configured on the qr interface of this router.

6. The address translation of the packet is completed on this router, and the source address becomes the external network address of the network node on the qg port (floating ip uses the external network address allocated to vm on the qg port, and it is necessary to bind the fix ip and floating ip in advance). The packet is sent back to br-int and marked with the internal vlan number.

7. The packet leaves br-int and enters br-ex, removing the internal vlan number and typing the vlan number of the external ip.

8. Finally, access the public network with the help of the provider network, which also confirms that the provider network can only be of the vlan or flat type.

VII. Network Node HA

From the above, we know that the three-layer communication of the whole architecture needs to be completed by means of L3 Agent installed in the network node. In this way, the network node becomes a single point of hidden danger in the architecture. In order to solve this problem, it is necessary to make high availability to the network node. The way for L3 Agent to realize ha is to provide a vip by using the keepalive VRRP protocol, and at the same time, add a ha interface on the br-int and Router to realize heartbeat transmission. At this time, the internal component communication of the network node is as shown in the following figure:

From the deployment structure point of view, the route namespace interface is the subnet gateway address and the external network address (or floating ip address), so when keepalive is used, the network node architecture is as shown in the following figure:

this concludes the introduction of vm traffic in centralized network node mode.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.