Shulou(Shulou.com)06/01 Report--

Recently, when the WEB server netstat-nap was discovered, there were too many connections in TIME_WAIT state, with each WEB exceeding 20, 000 and peaking over 40, 000, causing the service response to slow down

WEB server runs PHP programs, which need to call back-end MemCache,mysql,Redis,mongodb and back-end http interface services, in which MemCache and http interface services are relatively large.

According to the results of BAIDU, the reason for the analysis is that PHP is CGI, and a page request may have dozens of internal calls. After the program is executed, the connection is disconnected. According to the network protocol, the disconnected connection is bound to remain TIME_WAIT for a period of time, and the default is 4 minutes. If the number of requests within 4 minutes is too high, if it exceeds 30,000, the number of local TCP ports will inevitably be insufficient (by default, the range of linux temporary port numbers is (32768pc61000), and the number of ports that can be called by the machine is about 30,000), resulting in blocking of the calling back-end service and slow response of the page.

Processing method: according to the above analysis, modify the kernel parameters of OS system, enable TIME_WAIT connection reuse, TIME_WAIT connection recycling, shorten connection retention time, and increase the number of available ports:

# vi / etc/sysctl.conf

Net.ipv4.tcp_tw_reuse = 1 # enable connection reuse

Net.ipv4.tcp_tw_recycle = 1 # enable connection recycling

Net.ipv4.tcp_fin_timeout = 30 # shortens connection retention time

Net.ipv4.ip_local_port_range=1024 65000 # increases the number of ports available. The default temporary port number range for linux is (32768pc61000)

# sysctl-p # modified parameters take effect

Note: there is a pit here. You need to confirm that net.ipv4.tcp_timestamps = 1 (this is the default value) before the parameter net.ipv4.tcp_tw_recycle = 1 is valid.

In addition, the parameter net.ipv4.tcp_tw_recycle = 1 cannot be enabled for the server provided by public network users. If enabled, it will cause other problems. For more information, please see (http://blog.sina.com.cn/s/blog_781b0c850100znjd.html)

After modifying the above parameters, the number of connections is greatly reduced!

Summary: from the above analysis, there are mainly too many internal calls, and basically short connections, resulting in a large number of TIME_WAIT

From the perspective of application design, we should try to use long connections, connection pooling, KEEPALIVE and other technologies to reduce the number of connections to the back end and improve the efficiency of connections, which can greatly reduce the number of connections and the problem will not exist!

Of course, in this case, the front-end calling program is the PHP program, and due to the characteristics of PHP itself (CGI test, after a page is processed, all related connections are broken), it is not good to use persistent connections, connection pooling, and KEEPALIVE technology; but it can still be used internally in this page request, and the number of connections can be reduced to some extent.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.