Shulou(Shulou.com)06/02 Report--

Editor to share with you the use of iPerf testing and troubleshooting UDP packet loss problem, I believe that most people do not know much, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

Use iPerf testing and check for UDP packet loss problems

Phenomenon description

After using the high-speed tunnel to connect two VPC network type ECS instances in the same region (Region), the UDP packet loss rate between the two instances is tested by iPerf. The packet loss occurs when the test bandwidth reaches more than 50 Mbps, and the packet loss rate increases with the increase of bandwidth. As shown below:

Analysis of problems

Assuming that the private IP of two ECS instances of network type are VPC ECS A (192.168.104.235) and ECS B (10.182.83.13), and use Netcat (NC) to listen and send UDP data packets, the communication link diagram between ECS instance An and instance B of network type is as follows:

The direction of its data flow is as follows:

ECS A (192.168.104.235)-> NC 1 (100.105.59.3)-> VGW (10.141.166.253)-> NC 2 (100.105.59.9)-> ECS B (10.182.83.13)

We need to investigate and analyze its links to find out the ultimate cause of packet loss.

Solution method

Note: since you only see the communication before the source Netcat (that is, NC 1) and the destination Netcat (that is, NC 2), you should avoid the misunderstanding of packet capture and troubleshooting, that is, you should randomly judge that it is the direct communication between Netcat (NC) and lose packets.

When checking, you will find that the grab packet of the source eth0 is sent to the VGW, but the shell of the grab packet at the destination end is found to encapsulate the destination NC 2 IP, as shown in the example:

[Time] 17input 32input 07.130844 Point: `input `[ETHER] 24:4c:07:33:0e:02-> 00VAND 0437VV 280059.3-> 10.141.166.253 proto: 17, ver: 04, ihl: 05, len: 1534, ident: 59824 R: 0, DF: 1, MF: 0, offset: 0, ttl: 60, chksum: 0xfe47 [UDP] sport: 46703, dport: 250, size: 1514 Chksum: 0x0000 [VxLan] debug_flag: 0, vlan_tag: 0, payload_type: 0, version: 1, tunnel_id: 1878597, tos: 0, tof: 0 [IPv4] 192.168.104.235-> 10.182.83.13 proto: 17, ver: 04, ihl: 05, len: 1498, ident: 55469 Magic R: 0, DF: 1, MF: 0, offset: 0, ttl: 64, chksum: 0xd50e [UDP] sport: 36687, dport: 5001, size: 1478 Chksum: 0xa0aa [Time] 17 Time 32 Point 07.130854 Point: `output` [ETHER] 24:4c:07:33:0e:02-> 004V 04V 37281400V 65, eth_type: 0x0800 [IPv4] 100.105.59.3-> 100.105.59.9 proto: 17, ver: 04, ihl: 05, len: 1534, ident: 59824, R: 0, DF: 1, MF: 0, offset: 0, ttl: 60, chksum: 0x0000 [UDP] sport: 46703, dport: 250,size: 1514 Chksum: 0x0000 [VxLan] debug_flag: 0, vlan_tag: 0, payload_type: 0, version: 1, tunnel_id: 2125861, tos: 0, tof: 0 [IPv4] 192.168.104.235-> 10.182.83.13 proto: 17, ver: 04, ihl: 05, len: 1498, ident: 55469 Magic R: 0, DF: 1, MF: 0, offset: 0, ttl: 64, chksum: 0xd50e [UDP] sport: 36687, dport: 5001, size: 1478, chksum: 0xa0aa

After confirming that the packet has passed the VGW, start counting the packet capture information:

ECS A calls UDP traffic through iPerf: iperf-c 10.182.83.13-u-b 600m

ECS B receives through iPerf: iperf-u-s

Grab the package inside the instance.

ECS A:sudo tcpdump-w / client.pcap-n-I eth0 src host 192.168.104.25 and src port 1234ECS B:sudo tcpdump-w / server.pcap-n-I eth0 src host 192.168.104.25 and src port 1234

Grab the bag at the two NC eth0.

NC 1:sudo houyi-tcpdump-w / apsara/i-6we6pnh29n2q7srkgomd.pcap-nnK-I eth0 udp and src inner_port 1234 and dst inner_host 10.182.83.13NC 2:sudo houyi-tcpdump-B 4096-w / apsara/i-6we53i9h4ducbju5rmuw.pap-nn-I eth0 udp-K and src inner_host 192.168.104.235 and src inner_port 1234

Deploy the flow system in ASW and LSW.

100.105.59.3 purl 46728-> 10.141.166.253 purl 250

Note: because the destination NC 1 IP is automatically encapsulated in the destination packet shell, the message format of the VGW packet is: 100.105.59.3 pur46728-> 100.105.59.9 purl 250.

According to the analysis of the bag grabbing results.

ECS A packet loss / delivery: 171Comp510203

NC 1 eth0 package: 510204

ASW and LSW flow statistics package: 510204

NC 2 eth0 package: 510204

ECS B package: 510204 focus capture 507442, dropped by kernel 2162

The above analysis locates the packet loss of the instance protocol stack. Adjust the network stack (Stack) by adjusting the internal UDP Buffer Sizes of the instance. The default UDF Buffer Size is 212992 (208KB), and you can adjust it to 2097152 (2 MB).

/ proc/sys/net/core/rmem_default # default received packet memory size / proc/sys/net/core/rmem_max # maximum received packet memory size

Test UDP packet loss after adjustment.

The above is all the contents of the method of using iPerf to test and troubleshoot UDP packet loss, thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.