Shulou(Shulou.com)06/02 Report--

1. DVR experimental environment

A set of openstack environment in vlan mode. Interface_driver uses openvswitch and adds br-ex to the compute node for the virtual machine of the compute node to go out of the network.

II. DVR configuration

1. Control node

A, Editor / etc/neutron/neutron.conf

[DEFAULT]

Router_distributed = True

B. Restart the neutron-server service

2. Network node

A, Editor / etc/neutron/l3_agent.ini

[DEFAULT]

Agent_mode = dvr_snat

B, editor

/ etc/neutron/plugins/ml2/openvswitch_agent.ini

[agent]

Enable_distributed_routing = True

C. Restart neutron-l3-agent and neutron-openvswitch-agent services

3. Computing node

A, Editor / etc/sysctl.conf

Net.ipv4.ip_forward=1

Net.ipv4.conf.all.rp_filter = 0

Net.ipv4.conf.default.rp_filter = 0

B. Execute the command sysctl-p

C, Editor / etc/neutron/l3_agent.ini

[DEFAULT]

Interface_driver=neutron.agent.linux.interface.OVSInterfaceDriver

External_network_bridge =

Agent_mode = dvr

D, Editing

/ etc/neutron/plugins/ml2/openvswitch_agent.ini

[agent]

Enable_distributed_routing = True

E. Restart neutron-l3-agent and neutron-openvswitch-agent services

Third, enable dhcp to realize metadata function

1. By default, the metadata function is implemented through L3. In dvr mode, if you use L3, you need to configure additional metadata agent. For simplicity, this experiment enables dhcp agent to implement metadata function.

A. Edit / etc/neutron/dhcp_agent.ini on the network node

[DEFAULT]

Force_metadata = True

B. Restart the neutron-dhcp-agent service.

IV. Analysis of DVR to Gateway flow Table

1. Create a router, associate it with the test1 network (gateway 192.168.1.1), and create an instance (192.168.1.9) within the test1 network. The network model is as follows:

After doing this, neutron creates a namespace of router on both the network node and the compute node (with instance running), and the configuration is exactly the same.

In other words, there will be multiple identical router namespace in the platform, so how does neutron realize that virtual machine-to-gateway traffic only goes to the local router namespace? The answer is flow table.

2. For example:

A. In the ping gateway within the test1 (192.168.1.1), the virtual machine first sends an arp request to learn the mac address of the gateway. The arp message will enter the table0 of the br-int from the qvo-xxx port to match arp and in_port. According to the flow table entry, resubmit to table24.

B. Match arp, in_port and arp_spa in table24, and after hit, resubmit to table25.

C. Match in_port and dl_src in table25, and after hit, resubmit to table60.

D. In table60, hit table-miss flow entry and execute NORMAL according to action.

E. The gateway address is configured on qr-xxx in router namespace, while qr-xxx is on br-int.

Port, so the qr-xxx will receive the arp request and proceed accordingly, and the arp message will enter the br-int table0 again from the qr-xxx port and hit the table-miss flow entry, which will be forwarded by NORMAL and forwarded to the qvo-xxx of the virtual machine.

F, after the virtual machine learns the mac address of the gateway, the icmp message is forwarded to the gateway through the flow table, and the ping process is completed.

3. Now analyze how neutron realizes that the traffic from virtual machine to gateway only goes to the local router namespace through flow table.

In step d above, the table-miss flow entry is hit and the NORMAL is executed according to the action. The message may be forwarded normally to the br-vlan.

A. When the message arrives at br-vlan, it will enter table=0, match in_port, and then resubmit to table1 after hit.

B. In table1, match dl_vlan, arp_tpa (arp message) or dl_dst to drop all the messages to the gateway.

C. Through the above flow table, it is ensured that all messages arriving at the gateway are terminated locally in the host.

Fifth, the analysis of virtual machine intranet communication

In DVR mode, there are four scenarios for private network communication of virtual machines: same network host, different network host, different network host, and different network host. The following four scenarios are analyzed one by one:

1. Host on the same network does not need to go through router. The experimental model is as follows:

Test1 and test2 run on the same host and execute test1 ping test2.

A, test1 learns the mac address of test2 through arp, and the flow table matching process is the same as arp learning in step 4 (DVR to gateway flow table analysis). Then encapsulate the icmp message, enter the table0 of br-int from qvo-xxx port, match in_port, and resubmit to table25 after hit.

B. In table25, match in_port and dl_src, and after hit, resubmit to table60.

C. In table60, hit table-miss and perform NORMAL forwarding.

The qvo-yyy port of d and test2 is also on br-int, and the process of sending a message to test2,test2 via qvo-yyy and returning a packet to test is the same as from test1 to test2. The ping process ends.

2. Different hosts on the same network do not need router. The experimental model is as follows:

Test1 and test2 run on different hosts and execute test1 ping test2.

In this scenario, the steps are exactly the same as those in the same network homestay scenario. The message will execute NORMAL in the table60 of br-int. I will not repeat it here.

D. After executing NORMAL, the message will enter the table0 of br-vlan from phy-br-vlan port, match in_port, and then resubmit to table1 after hit.

E. In table1, hit table-miss,resubmit table2.

F. In table2, match in_port and dl_vlan. After a hit, modify local vlan 1 to global vlan 196, and then execute NORMAL. At this time, the message will leave the host and be sent out the service port.

G, the message will be sent to the host business port running test2, and then enter the table0 of br-vlan, and after hit, resubmit to table3.

H. In table3, after a hit, the NORMAL is executed and the message is sent to br-int.

I. In the table0 of br-int, match in_port and dl_vlan. After the hit, modify the global vlan 196 to local vlan 1, then execute NORMAL, and the message will enter the virtual machine test2 from qvo-yyy.

J and test2 return packets are the same as test1 sending packets. At this point, the ping operation is completed.

3. Router is required for different network hosts. The experimental model is as follows:

Create a router to connect two networks, and each one on the network creates an instance. After performing the above operations, neutron will create a router namespace on both the network node and the compute node (with instance running), and the configuration is exactly the same. The qr- device will be configured with the corresponding gateway ip address.

Test1 and test2 run on the same host and execute test1 ping test2.

A, test1 knows through calculation that it needs to go three layers to reach test2. It will first send an arp request, learn the mac address of the gateway, then encapsulate the icmp message (the destination ip is the ip of test2, and the destination mac is the mac address of the gateway), and send the message to the local router namespace through the default route.

(please refer to the section for matching flow tables in the process of learning gateways.)

B, the message enters the PREROUTING chain of router from qr-test1, and then finds the route of test2. After hitting, it enters the POSTROUTING chain, and re-enters the table0 of br-int from qr-test2.

C. In table0, hit table-miss,resubmit to table60.

D. In table60, after a hit, execute NORMAL and send the message from qvo-xxx to test2.

E. The process of returning test2 packets is the same as that of test1 sending packets. At this point, the ping operation is completed.

4. Router is required for different hosts in different networks. The experimental model is as follows:

Test1 and test2 run on different hosts and execute test1 ping test2.

In this scenario, the steps are exactly the same as those for homestay scenarios on different networks.

After the message comes out of the qr-test2, the NORMAL is executed in the table60 of the br-int, which is not repeated here.

D. After executing NORMAL in c, the message enters the table0 of br-vlan through int-br-vlan port.

E. In table0, match in_port, and after hit, resubmit to table1.

F. In table1, match dl_vlan and dl_src. After a hit, change the source mac address to neutron and assign it to host mac,resubmit to table2.

Note: when DVR is enabled, neutron assigns a unique mac address to each compute node to avoid mac address conflicts on physical switches.

G. In table2, match in_port and dl_vlan. After hitting, modify local vlan 2 to global.

Vlan 148executes NORMAL, and the message leaves the host from the service port eth2.

The message will enter the business port eth2 of the host running test2 and enter the table0 of br-vlan.

I, in table0, after hitting, resubmit to table3.

J. In table3, match dl_src (the mac assigned by neutron to the host), and send the message from phy-br-vlan to br-int table0.

K. In the table0 of br-int, match in_port and dl_src (the mac assigned by neturon to the host), and resubmit to table2.

L. In table2, match dl_vlan and dl_dst, and modify the source mac to mac,resubmit to table60 of the test2 gateway.

M. In table60, match dl_vlan and dl_dst, peel off the vlan, and send the message directly to test2 through output.

N. The process of returning a test2 packet is the same as that of sending a test1 packet. At this point, the ping operation is completed.

VI. Analysis of the principle of virtual machine going out of the network.

1. Create a router, bind the internal network test1, set the gateway of the router to external, and create a virtual machine on the internal network and the external network for testing. The experimental model is as follows:

After doing this, neutron creates three namespace:qrouter-xxx, fip-yyy, and snat-xxx on the network node.

The compute node (instance runs) creates qrouter-xxx and fip-yyy.

Ps: the namespace of each node will explain its role in the following analysis.

2. The message of the virtual machine test1 is sent to the qr-xxx device of the host router namespace (explained above, but will not be repeated here), enters the PREROUTING chain (missed, without any modification), checks the policy route, uses the default route to prepare for forwarding, and then enters the POSTROUTING chain (missed). The message is sent from qr-xxx to 192.168.1.11 for processing (the flow table all hits NORMAL).

3. Some people may wonder, what is 192.168.1.11 here?

In fact, this ip address is an ip assigned by neutron to snap-yyy namespace, which is configured on sg-zzz. Snap-yyy namespace is on the network node, and neutron forwards the message in the computing node router namespace to the snat-yyy of the network node through policy routing and sg-zzz port.

4. When the message arrives on the sg-zzz port in the snat-yyy namespace of the network node, the message will enter the PREROUTING chain (missed) before routing, and then look for the route.

Then enter the POSTROUTING chain, snat, and do connection tracking, and then the message is forwarded.

5. After the above operation, the virtual machine goes out of the external network through the qg- device in snat-yyy and returns through connection tracking (the flow table hits NORMAL).

VII. Analysis of Floating ip principle.

1. On the basis of Section 6, assign a floating ip (10.100.0.7) from the public network and associate it to the virtual machine test1 (192.168.1.9). You need to configure br-ex for the virtual machine to go out to the public network. The experimental model is as follows:

After performing the above operations, neutron will add several policies to the corresponding namespace, which are described one by one in the following analysis:

1. The message of the virtual machine test1 is sent to the qr-xxx device of the host router namespace (explained above, but will not be discussed here), enter the PREROUTING chain (missed, without any modification), and check the policy route.

Through policy routing, the message is forwarded to 169.254.109.47 through rfp-6347c62b-2 for processing, and then enters the POSTROUTING chain to make snat modification.

2. At this point, you may have a question: what is this rfp-6347c62b-2 device?

In fact, this is one end of veth pair, and the other end of it is in fip-xxx namespace.

Neutron uses this pair of veth pair to forward messages from the namespace of router to the namespace of fip-yyy through policy routing.

3. At this point, the message arrives at fip-yyy namespace, enters the PREROUTING chain (missed), and checks the route.

Then enter the POSTROUTING chain (missed) to forward the message from the fg-7ec56cee-b5 device to the external network.

4. At this point, the message is successfully sent from fg port to the public network (the flow table hits NORMAL).

5. Now begin to analyze the situation in which the external network devices enter the virtual machine through floating ip.

Suppose that there is an external network device external (10.100.0.14) to ping floating ip (10.100.0.7), and the external network device will first learn the mac address of 10.100.0.7 through arp, but in the above description, neutron does not configure the ip address of 10.100.0.7 on any device, that is to say, 10.100.0.7 does not exist, how can the message be accurately sent to the fg port?

In fact, neutron turns on the arp_haproxy function on fg port, which is equivalent to arp spoofing, so that the external network device will learn the mac address of fg to 10.100.0.7 mac and update it to its own mac table.

6. After the public network message arrives at the fg port, enter the PREROUTING chain (missed), check the route table, and prepare to send the message with the destination ip of floating ip from fpr-6347c62b-2 to 169.254.109.46.

It then enters the POSTROUTING chain (missed) and the message is forwarded to the namespace of router.

7. The message arrives at the rfp-6347c62b-2 device in the namespace of router, enters the PREROUTING chain, and performs dnat operation. Convert the destination ip from floating ip to internal fix ip (192.168.1.9)

Then check the route and prepare to forward the message from the qr-xxx port.

It then enters the POSTROUTING chain (missed) and forwards the message from qr- port to the virtual machine test1.

8. At this point, the forwarding of the external network device to the internal ip is completed.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.