Shulou(Shulou.com)05/31 Report--

In this issue, Xiaobian will bring you an analysis of how to achieve Linux 2.6.16 TCP connection speed anomalies. The article is rich in content and analyzed and described from a professional perspective. After reading this article, I hope you can gain something.

It was found that access to some of the company's businesses was very erratic and generally slower than competitors. Analysis shows that there is incompatibility between kernel TCP congestion control algorithm on SESU10 master disk and Ack frequency control strategy of Windows.

At least 2.6.16 kernel version is confirmed to have this problem; TCP optimization patch or replacement of Tlinux can solve the problem.

Problem phenomena:

Test in the experience network environment: In the case of large file downloads, Baidu's download speed averages 600KBPS, and our download speed averages less than 100Kbps; in the case of interactive Webgame, TNT service download speed is about 25% of DDT.

Here is a typical download speed curve:

Curve of our server: (vertical axis: packets/s)

Baidu server download curve:

Test environment to reproduce the problem:

Network: Company experience network, ordinary Unicom 4M ADSL

Server: Linux64-bit server, Shenzhen computer room.

Server program: Apache, nws (self-developed webserver)

Client: Windows XP, Windows 7, any browser or Cyclone (single threaded download)

Testing tools: wireshark, httpwatch

Test connection: self-built CDN, Baidu download, Shenzhen DC+Apache

Problem analysis:

Client-side packet capture analysis found that slow segments have two problems:

The server always waits until the previous packet is acknowledged before sending the second packet

Windows always waits around 200ms to send an ACK acknowledgement.

For Windows behavior, to prevent excessive ACKs from causing network stress, the Ms TCP stack starts a 200ms timer on each packet received and does not send an ACK packet until another packet is received or the timer expires.

After turning off Ack delay by setting the registry option TcpAckFrequency parameter to 1, the experiment found that the download speed returned to normal, and the problem of slow download speed could not be reproduced.

To configure the max outstanding ACKs in Windows XP/2003/Vista/2008:

[HKEYLOCALMACHINE \SYSTEM \CurrentControlSet \Services \Tcpip \Parameters \Interfaces \{Adapter-id}]

TcpAckFrequency = 1 (Default=2, 1=Disables delayed ACK, 2-n = If n outstanding ACKs before timed interval, sent ACK)

Because users cannot be forced to modify the registry to avoid problems, and competitors do not see similar problems, they can only be solved from the linux side.

On the Linux side, it is suspected that it has something to do with the Nagle algorithm. After setting TCP_NODELAY on the nws server, it can still be reproduced, and the influence of the Nagle algorithm can be ruled out. (In fact, every time nws sends a large packet or directly uses sendfile, it will not be affected by nagle algorithm.) Secondly, Apache and nws can reproduce this problem, and it is more suspected that the operating system itself has defects.

Since Linux only sends one packet at a time, the congestion window problem is suspected, and the problem is speculated to be as follows:

In the initial case, when the client replies an ACK, the congestion window increases, and multiple packets are sent each time, so the transmission speed can be faster at the beginning; because the network delay jitter or packet loss causes the server protocol stack to determine that the packet timeout, the congestion window is reset to 1, only one packet is sent each time, and when the client 200ms packet reply is received, it is still considered as timeout, and RTT is adjusted at the same time; Until RTT increases to 200ms, the congestion window expands, multiple packets can be sent, transmission speeds increase, and so on.

The download speed returned to normal by testing to increase the initial congestion window to 10 (implemented by replacing the TCP optimization module of the kernel loader leveling new technology group).

The above is a small series for everyone to share how to achieve Linux 2.6.16 TCP connection speed abnormal problem analysis, if there is just a similar doubt, may wish to refer to the above analysis to understand. If you want to know more about it, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.