Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to solve the problem of Ulimit". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to solve the problem of Ulimit.

I have encountered a very interesting question recently. Among them is a group of HAProxy, which has frequent problems. Log in to the server and check cpu, memory, network and io. It turns out that there are more than 60, 000 connections in the TIME_WAIT state on the machine.

TIME_WAIT status generally appears on proxy machines such as HAProxy and Nginx, mainly due to frequent active shutdown. By modifying reuse and recycling parameters, the problem can be solved more quickly.

The number of network status statistics can be calculated using the following command.

Netstat-ant | awk'/ ^ tcp/ {+ + S [$NF]} END {for (an in S) print (AME S [a])} 'ESTABLISHED 70 FIN_WAIT2 30 CLOSING 33 TIME_WAIT 65520

It's not magical, but the number 65535 is too sensitive. Must have triggered some kind of upper limit.

What makes us even more puzzled is: why is the service unavailable when the connection in the TIME_WAIT state reaches only 65535?

Are you bragging about the million connections of a single machine everywhere? Why can't you stand the trouble?

65535, which is equal to 2 to the power of 16 minus one, is a magic number. Putting this small number aside, let's see how many connections Linux can support.

1. How many connections can Linux support?

The answer is countless. But there are only 65535 ports.

Why are there only 65535 ports?

This is a historical reason, because at the beginning of the TCP and UDP protocols, there are 16 bits to store the source port number and the destination port number respectively. Unfortunately, this value is of type short, and the size is also 2 ^ 16-1.

The immutable standard caused by historical reasons is so deep-rooted.

So how many connections can Linux support? The answer is countless.

Take nginx, for example, we monitor it on port 80. At this time, machine A connects to Nginx and can initiate as many as 6w long-term connections. If machine B connects to Nginx, it can also initiate more than 6w connections. This is due to the determination of a connection, which is jointly determined by src and dst.

The idea that Linux can only accept 65535 connections can only be said to be a very simple presumption.

65535 ports, which may be too small for you as a pressure measuring machine. But for the server, it's more than enough.

two。 How to support millions of connections?

As you can see from the above, there is no limit to the number of connections. But Linux also has a layer of protection, and that is the number of file handles. What you see through the lsof command is the so-called file handle.

First, let's take a look at the display of several commands.

Ulmit, which shows the number of file handles each process can occupy.

Ulimit-n 65535

File-max, which shows the total number of file handles that the operating system can occupy, for all processes.

Cat / proc/sys/fs/file-max 766722

File-nr, which shows the number of handles that have been used and the total number of handles. It can be used for surveillance.

Cat / proc/sys/fs/file-nr 1824 0 766722

To support millions of connections, release both operating system-level and process-level handles. In other words, the display of ulimit and file-max must be greater than one million.

3. How to set it up?

The common way to set the number of handles to a process is ulimit, but it is not recommended. For no other reason, the ulimit setting takes effect only if the process is started in the same shell. If you open another shell or restart the machine, the changes to ulimit will be lost. This is the way:

Ulimit-n 1000000

The correct way is to modify the / etc/security/limits.conf file. For example, the following content.

Root soft nofile 1000000 root hard nofile 1000000 * soft nofile 1000000 * hard nofile 1000000

As you can see, we can modify the number of handles for a specific user. This is often encountered when installing applications such as es.

Es-nofile 65535

But even in this way, you need to open a new shell to operate. It also does not take effect in the current modified shell or in the shell before the modification. Xjjdog has encountered a number of cases in which restrictions have been relaxed but problems have occurred.

To see if these changes have taken effect on the process, you can look at the process's memory-mapped file. Cat / proc/180323/limits, for example, will be shown in detail.

This number is not as big as you want. The upper limit of its size is determined by nr_open. To get bigger, change the value of fs.nr_open in / ect/sysct.conf.

Cat / proc/sys/fs/nr_open 1048576

So how to modify file-max? It is recommended to modify the / etc/sysctl.conf file by adding the following. There are more than 6 million!

Fs.file-max = 6553560

When the number of files exceeds, an error of kernel: VFS: file-max limit 65535 reached is reported.

To sum up.

Even if Linux opens a port, it can accept a large number of connections. The upper limit of these connections is limited by the number of file handles in a single process and the number of operating system file handles, namely ulimit and file-max.

In order to persist the parameter changes, we tend to write the changes to the file. The file handle limit of the process can be placed in / etc/security/limits.conf, and its upper limit is restricted by fs.nr_open; the file handle limit of the operating system can be placed in the / etc/sysctl.conf file. Finally, don't forget to confirm that the changes are in effect for the process in the / proc/$id/limits file.

At this point, I believe you have a deeper understanding of "how to solve the problem of Ulimit". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.