In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-28 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/01 Report--
This article introduces the relevant knowledge of "what is the TCP full connection queue". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!
First, problems
Today, a little friend came up to me and told me that there was a strange problem that needed help, and the problem was really strange. The client-side call RT is relatively high and appears with intermittent exception Connection reset, while the server-side CPU, thread stack, and so on all look normal, and the server-side RT is very short.
Let's start with the result: the connection caused by the small TCP full connection queue is discarded because the project uses Spring Boot's built-in Tomcat, and the default accept-count is 100. in this case, this parameter represents the full connection queue size. Therefore, when the request peak, the full connection queue is full, which leads to the discarding of connections. So we adjusted the parameter server.tomcat.accept-count to solve the problem.
2. Semi-connected queue and fully connected queue
Well, in order to know why, from the abnormal information, it may be what is wrong with the TCP connection, which focuses on the semi-connection queue and the full connection queue. Let's take a look at what TCP semi-connected queues and fully connected queues are and why this strange phenomenon occurs.
1. TCP three-way handshake process and queue
During the TCP three-way handshake, the Linux kernel maintains two queues:
Semi-connected queues, called SYN queues
Fully connected queues, called accept queues
It's a clich é. Let's start with the fact that everyone is familiar with the three-way handshake of TCP. Let's look at a picture:
1. The client sends a SYN packet and enters the SYN_SENT state.
2. The server receives the data packet and puts the relevant information into the semi-connection queue (SYN queue), and returns the SYC+ACK packet to the client.
3. When the server receives client ACK packets, if the fully connected queue (accept queue) is not full, it will take the data out of the semi-connected queue and put it into the fully connected queue, waiting for the application to use it. When the queue is full, the policy will be executed according to the tcp_abort_on_overflow configuration.
Here semi-connected queues (SYN queues) and fully connected queues (accept queues) are the key points.
2. View the full connection queue
When querying a question, we need to check the status of the fully connected queue. On the server side, we can use the ss command to view, and the ss command to obtain data is divided into LISTEN status and non-LISTEN status.
Data in LISTEN status:
#-l shows that the socket#-n in Listener does not resolve the service name #-t only shows the total number of TCP full connections that the tcp# Recv-Q has completed the three-way handshake and waited for the server accept (). # Send-Q full connection queue size [root@server ~] # ss-lnt | grep 6080State Recv-Q Send-Q Local Address:Port Peer Address:PortLISTEN 0 100:: 6080:: *
Data in non-LISTEN state:
# number of bytes received by Recv-Q but not read by application process # number of bytes sent by Send-Q but not received acknowledgment [root@server ~] # ss-nt | grep 6080State Recv-Q Send-Q Local Address:Port Peer Address:PortESTAB 0433:: 6080:: * 3, full connection queue overflow
When a large number of requests enter, if the TCP full connection queue is too small, the full connection queue will overflow. When the full connection queue overflow occurs, the subsequent requests will be discarded, and the number of service requests will not go up.
As mentioned earlier, in the last step of the TCP three-way handshake, when the full connection queue is full, it will be processed according to the tcp_abort_on_overflow policy. Linux can be configured through / proc/sys/net/ipv4/tcp_abort_on_overflow.
When the tcp_abort_on_overflow=0, the service accept queue is full, the client sends the ack, and the server discards the ACK directly. At this time, the server is in the state of [syn_rcvd] and the client is in the state of [established]. In this state, a timer will retransmit the server SYN/ACK to the client (no more than the number of times specified by / proc/sys/net/ipv4/tcp_synack_retries. The default is 5 under Linux). After that, the server will not retransmit, and there will be no subsequent action. If the client sends data at this time, the server will return RST. (this is the cause of our anomaly.)
When the tcp_abort_on_overflow=1, the server accept queue is full, the client sends ack, and the server directly returns RST to notify client, indicating that the handshake process and the connection are cancelled, and the client will report to connection reset by peer.
1)。 When the full connection queue overflows, what indicators can be explained, and what effective query methods do we have?
Command query, we can look at the handshake features of TCP:
[root@server ~] netstat-s | egrep "listen | LISTEN" 7102 the number of full connection queue overflows 7102 SYNs to LISTEN sockets ignored indicates the number of semi-connection queue overflows 2times represents the number of full connection queue overflows. Query every few seconds. If this number keeps increasing, it indicates that the full connection queue has been overflowed. Configure fully connected queues and semi-connected queues?
The full connection queue size depends on the minimum value of backlog and somaxconn, that is, min (backlog,somaxconn)
Somaxconn is a Linux kernel parameter. It defaults to 128and can be configured via / proc/sys/net/core/somaxconn.
Backlog is a parameter in the listen (int sockfd,int backlog) function. Backlog,Tomcat defaults to 100.
The length of the semi-connection queue can be set through / proc/sys/net/ipv4/tcp_max_syn_backlog. OS layer, only one can be set, which is shared by all programs.
3)。 View semi-connection status
The semi-connection, that is, the TCP connection in the SYN_RECV state of the server, is in the semi-connection queue, so you can use the following command to calculate:
# check the semi-connection queue [root@server ~] netstat-natp | grep SYN_RECV | wc-l233 # indicates that the semi-connected TCP connection has 233 "what is the TCP full connection queue". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 278
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.