Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to understand TCP's SYN queue and Accept queue". In daily operation, I believe many people have doubts about how to understand TCP's SYN queue and Accept queue. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "how to understand TCP's SYN queue and Accept queue"! Next, please follow the editor to study!

About two queues

First of all, we must understand that the TCP socket in the "LISTENING" state has two separate queues:

SYN queue (SYN Queue)

Accept queue (Accept Queue)

These two terms are sometimes referred to as "reqsk_queue", "ACK backlog", "listen backlog" and even "TCP backlog", but we use the above two terms in this article to avoid confusion.

SYN queue

The SYN queue stores the connection that received the SYN packet (the corresponding kernel code structure: struct inet_request_sock). Its job is to reply to the SYN+ACK packet and retransmit it when no ACK packet is received until it times out. Under Linux, the number of retransmissions is as follows:

$sysctl net.ipv4.tcp_synack_retriesnet.ipv4.tcp_synack_retries = 5

The description of tcp_synack_retries in the document is as follows:

The number of times the tcp_synack_retries-int integer retransmits the SYNACKs for a passive TCP connection. The value cannot exceed 255. The default value is 5, and if the initial RTO is 1 second, the corresponding last retransmission is 31 seconds. The corresponding last timeout is 63 seconds later.

After sending the SYN+ACK, the SYN queue waits for the ACK packet sent from the client (that is, the last packet of the three-way handshake). When receiving the ACK packet, first find the corresponding SYN queue, and then check the relevant data in the corresponding SYN queue to see if it matches. If so, the kernel removes the data related to the connection from the SYN queue, creates a complete connection (corresponding to the structure of the kernel code: struct inet_sock), and adds the connection to the Accept queue.

Accept queue

What is stored in the Accept queue is the established connection, that is, the connection waiting to be removed by the upper application. When the process calls accept (), the socket is taken from the queue and passed to the upper application.

This is a simple description of how Linux handles SYN packages. By the way, when socket turns on TCP_DEFER_ACCEPT and TCP_FASTOPEN, there will be a slight difference in how it works, which is not covered in this article.

Queue size limit

The application sets the maximum size of the SYN queue and the Accept queue by calling the system call listen (2), passing in the backlog parameter. For example, set the maximum size of both the SYN queue and the Accept queue to 1024:

Listen (sfd, 1024)

Note that prior to version 4. 3 of the kernel, the size of the SYN queue was calculated in a different way.

The maximum size of the SYN queue used to be configured with net.ipv4.tcp_max_syn_backlog, but it is no longer used. Net.core.somaxconn is now used to represent the maximum size of both the SYN queue and the Accept queue. On our server, we set it to 16k:

$sysctl net.core.somaxconnnet.core.somaxconn = 16384 queue is set to an appropriate size

After knowing the above information, you may ask, how big is the queue set?

The answer is: it depends. For most TCP services, this is not very important. For example, prior to version 1.11 of the Go language, there was no way to set the queue size.

However, there are some reasonable reasons to increase the size of the queue:

When the speed of requests to establish a connection is indeed high, the SYN queue may need to be set up larger, even for a high-performance service.

The size of the SYN queue, in other words, is the number of connections waiting for the ACK packet. That is, the greater the average round-trip time to the client, the more connections are accumulated in the SYN queue. For those scenarios where most clients are far away from the server, for example, the round trip time is more than a few hundred milliseconds, you can set the queue size a little larger.

If the TCP_DEFER_ACCEPT option is turned on, it causes the socket to stay in the SYN-RECV state for longer, that is, to increase the time spent in the SYN queue.

However, setting the backlog too large can also have a bad effect:

Each slot in the SYN queue takes up some memory. When it comes to SYN Flood attacks, there is no need to waste resources on these attacking packets. The inet_request_sock structure in the SYN queue, under the 4.14 kernel, each takes up 256 bytes of memory.

Under linux, if you want to see the current status of the SYN queue, we can use the ss command to query the socket of the SYN-RECV status. For example, the following execution result indicates that there are currently 119 elements in the SYN queue for port 80 and 78 for port 443.

$ss-n state syn-recv sport =: 80 | wc-l119 $ss-n state syn-recv sport =: 443 | wc-l78

You can also look at this data through our SystemTap script: resq.stp

What if the program doesn't call accept () fast enough?

What happens if the program doesn't call accept () fast enough?

Subsequent SYN packets received will not be processed by the SYN queue

Subsequent ACK packets received (for establishing a connection) will not be processed by the SYN queue

TcpExtListenOverflows / LINUX_MIB_LISTENOVERFLOWS count increased

TcpExtListenDrops / LINUX_MIB_LISTENDROPS count increased

When this happens, we can only hope that the processing performance of the program will return to normal later, and the client will resend the discarded packet by the server.

This performance of the kernel is acceptable for most services. By the way, you can modify this behavior by adjusting the global parameter net.ipv4.tcp_abort_on_overflow, but it's best not to change this parameter.

You can observe the status of Accept queue overflows by looking at the nstat count:

$nstat-az TcpExtListenDropsTcpExtListenDrops 49199

But this is a global count. The observation is not intuitive enough, for example, sometimes we observe that it is growing, but all the services seem to be normal. At this point, we can use the ss command to observe the Accept queue size of a single listening port:

$ss-plnt sport =: 6443 | catState Recv-Q Send-Q Local Address:Port Peer Address:PortLISTEN 0 1024 *: 6443 *: *

The Recv-Q column shows the number of socket in the Accept queue, and the Send-Q shows the maximum size of the queue. In the above example, we found that there is no socket without the program accept (), but we still find that the ListenDrops count is growing.

This is because our program is periodically temporarily stuck without handling new connections, not permanently, and the program returns to normal after a period of time. In this case, it is difficult to observe this phenomenon with the ss command, so we wrote a SystemTap script that hook into the kernel and prints out the discarded SYN packets:

$sudo stap-v acceptq.stptime (us) acceptq qmax local addr remote_addr1495634198449075 1025 1024 0.0.0.0Viru 6443 10.0.1.92Discovery 285851495634198449253 1025 1024 0.0.0.0Partition 6443 10.0.1.92PLO 5050014956341984500621025 1024 0.0.0.0Virus 6443 10.0.0.92 acceptq qmax local addr remote_addr1495634198449075 65434.

Through the above operation, you can observe which SYN packets are affected by ListenDrops. So we can know which programs are losing connections.

At this point, the study on "how to understand TCP's SYN queue and Accept queue" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.