Shulou(Shulou.com)05/31 Report--

This article mainly introduces the relevant knowledge of "how to optimize Nginx and Node.js for high-load networks". Xiaobian shows you the operation process through actual cases. The operation method is simple and fast and practical. I hope this article "how to optimize Nginx and Node.js for high-load networks" can help you solve the problem.

network tuning

If you don't understand the underlying transport mechanism of nginx and node.js first and optimize them accordingly, you may be able to tune them carefully in vain. In general, nginx connects clients to upstream applications through tcp sockets.

Our system has many thresholds and limits on tcp, set by kernel parameters. Default values for these parameters are often set for general use and do not meet the high traffic, short life requirements required by web servers.

Here are some of the parameters that are candidates for tuning tcp. To make them work, you can put them in the/etc/sysctl.conf file, or in a new configuration file such as/etc/sysctl.d/99-tuning.conf, and run sysctl -p to have the kernel load them. We use sysctl-cookbooks to do this manual work.

It should be noted that the values listed here are safe to use, but it is recommended that you study the meaning of each parameter in order to choose a more appropriate value according to your load, hardware and usage.

The copy code is as follows:

net.ipv4.ip_local_port_range='1024 65000'

net.ipv4.tcp_tw_reuse='1'

net.ipv4.tcp_fin_timeout='15'

net.core.netdev_max_backlog='4096'

net.core.rmem_max='16777216'

net.core.somaxconn='4096'

net.core.wmem_max='16777216'

net.ipv4.tcp_max_syn_backlog='20480'

net.ipv4.tcp_max_tw_buckets='400000'

net.ipv4.tcp_no_metrics_save='1'

net.ipv4.tcp_rmem='4096 87380 16777216'

net.ipv4.tcp_syn_retries='2'

net.ipv4.tcp_synack_retries='2'

net.ipv4.tcp_wmem='4096 65536 16777216'

vm.min_free_kbytes='65536'

Highlight some of the important ones.

net.ipv4.ip_local_port_range

In order to serve downstream clients for upstream applications, nginx must open two tcp connections, one to the client and one to the application. When the server receives a lot of connections, the available ports of the system will be exhausted quickly. The range of available ports can be increased by modifying the net.ipv4.ip_local_port_range parameter. If an error is found in/var/log/syslog: "possible syn flooding on port 80. "sending cookies" means that the system cannot find an available port. Increasing the net.ipv4.ip_local_port_range parameter reduces this error.

net.ipv4.tcp_tw_reuse

When the server needs to switch between a large number of tcp connections, a large number of connections in the time_wait state are generated. time_wait means that the connection itself is closed, but the resource has not been released. Setting net_ipv4_tcp_tw_reuse to 1 lets the kernel recycle connections as much as possible when it is safe, which is much cheaper than reestablishing new connections.

net.ipv4.tcp_fin_timeout

This is the minimum time a connection in the time_wait state must wait before recycling. Making it smaller speeds up recycling.

How to check connection status

Using netstat:

netstat -tan | awk '{print $6}' | sort | uniq -c

Or use ss:

ss -s

nginx

ss -s

total: 388 (kernel 541)

tcp: 47461 (estab 311, closed 47135, orphaned 4, synrecv 0, timewait 47135/0), ports 33938

transport total ip ipv6

* 541 - -

raw 0 0 0

udp 13 10 3

tcp 326 325 1

inet 339 335 4

frag 0 0 0

As the load on the web server increases, we start to encounter some strange limitations of nginx. Connection dropped, kernel keeps reporting syn flood. At this point, the average load and cpu usage are very low, and the server can obviously handle more connections, which is really frustrating.

After investigation, it was found that there were a lot of connections in time_wait state. Here is the output from one of these servers:

There are 47135 time_wait connections! And, as you can see from ss, they are all closed connections. This means that the server has consumed most of the available ports, and implies that the server is assigning new ports for each connection. Tuning the network helps a bit with this problem, but there are still not enough ports.

After further research, I found a document on the uplink keepalive directive that reads:

Sets the maximum number of idle keepalive connections to upstream servers that are retained in the worker process cache.

Interesting. In theory, this setup minimizes connection waste by passing requests over cached connections. The documentation also mentions that we should set proxy_http_version to "1.1" and clear the "connection" header. After further research, I found this to be a good idea because http/1.1 greatly optimizes tcp connection usage compared to http 1.0, which nginx uses by default.

After modifying as suggested in the document, our uplink profile looks like this:

The copy code is as follows:

upstream backend_nodejs {

server nodejs-3:5016 max_fails=0 fail_timeout=10s;

server nodejs-4:5016 max_fails=0 fail_timeout=10s;

server nodejs-5:5016 max_fails=0 fail_timeout=10s;

server nodejs-6:5016 max_fails=0 fail_timeout=10s;

keepalive 512;

}

I also modified the proxy settings in the server section as it suggested. Also, a proxy_next_upstream was added to skip failed servers, keepalive_timeout was adjusted for clients, and access logs were turned off. The configuration looks like this:

The copy code is as follows:

server {

listen 80;

server_name fast.gosquared.com;

client_max_body_size 16m;

keepalive_timeout 10;

location / {

proxy_next_upstream error timeout http_500 http_502 http_503 http_504;

proxy_set_header connection "";

proxy_http_version 1.1;

proxy_pass http://backend_nodejs;

}

access_log off;

error_log /dev/null crit;

}

With the new configuration, I noticed that the servers were using 90% fewer sockets. Requests can now be transmitted with far fewer connections. The new output is as follows:

ss -s

total: 558 (kernel 604)

tcp: 4675 (estab 485, closed 4183, orphaned 0, synrecv 0, timewait 4183/0), ports 2768

transport total ip ipv6

* 604 - -

raw 0 0 0

udp 13 10 3

tcp 492 491 1

inet 505 501 4

node.js

Thanks to its event-driven design for asynchronous i/o processing, node.js handles a large number of connections and requests out of the box. While there are other tuning tools, this article will focus primarily on the process aspects of node.js.

Node is single-threaded and does not automatically use multicore. That is, the application cannot automatically acquire the full capabilities of the server.

Clustering of node processes

We can modify the application so that it forks multiple threads and receives data on the same port, thus achieving load spanning multiple cores. Node has a cluster module that provides all the tools necessary to achieve this goal, but it takes a lot of physical effort to incorporate them into the application. If you are using express, ebay has a module called cluster2 that you can use.

Prevent context switching

When running multiple processes, make sure each cpu core is busy with only one process at a time. In general, if the cpu has n cores, we should generate n-1 application processes. This ensures that each process gets a reasonable slice of time, leaving the kernel scheduler to run other tasks. We also want to make sure that almost no tasks other than node.js are performed on the server to prevent cpu contention.

We made the mistake of deploying two node.js apps on the server and then opening n-1 processes for each app. As a result, they snatch cpu from each other, causing the system to load sharply. Although our servers are all 8-core machines, the performance overhead caused by context switching is still palpable. Context switching is when the cpu suspends the current task in order to perform another task. On switching, the kernel must suspend all state of the current process and then load and execute another process. To solve this problem, we reduced the number of open processes per app so that they shared the CPU fairly, resulting in a reduction in system load:

Notice how the system load (blue line) drops below the CPU core count (red line). On other servers, we see the same thing. Since the total workload remains the same, the performance improvement in the figure above can only be attributed to a reduction in context switching.

About "how to optimize Nginx and Node.js for heavily loaded networks" content is introduced here, thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the industry information channel. Xiaobian will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.