Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to deal with the Accept Emfile problem, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let Xiaobian take you to understand.

Typically, when the server calls the accept function, it returns a new file descriptor for data transfer to and from the client

In the development of the server, this situation is sometimes encountered: when the accept function is called to accept the client connection, the function returns a failure, and the corresponding error code is EMFILE, which indicates that the file descriptor opened by the current process has reached the upper limit. At this time, the server can no longer accept client connections.

When you encounter the above problems, how to deal with them reasonably, let's analyze them below.

The process of establishing a connection

Let's briefly review the process of establishing a connection between the client and the server, as shown in the following figure:

1. The client initiates a SYN request

two。 After the server receives the SYN request from the client, the kernel puts the connection into the semi-connection queue and returns a SYN + ACK to the client.

3. The client returns an acknowledgement ACK to the server. After the server receives this ACK, the three-way handshake is completed. At the same time, the kernel removes the connection from the semi-connection queue, creates a new full connection, and joins the full connection queue.

4. The application layer calls the accept function to extract the connection from the full connection queue.

Steps 1, 2, and 3 above are the three-way handshake of TCP, which is done by the TCP protocol in the kernel, and step 4 is the application layer calling the accept interface.

Problems in epoll

Epoll is an IO multiplexing model in Linux, which is widely used in server development. Here is an example of epoll.

After the server creates the listening file descriptor listenfd, registers the read event with epoll

When epoll detects a read event on the listenfd, it immediately notifies the application layer, and the application layer calls accept to accept the new connection. At this time, the number of file descriptors opened by the process has reached the limit, so each accept fails.

The following problems will arise here

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Because each accept fails, which is equivalent to the unhandled readable events on the listenfd, the epoll will constantly trigger the readable events on the listenfd, and the application layer will keep calling accept, and then the accept call will fail again. Such a continuous execution of invalid loops is a waste of CPU resources.

It is mentioned above that the server is constantly executing invalid loops, which will cause another problem. If a new client connection arrives at this time, the process of establishing the connection will be very slow.

The epoll mentioned earlier uses horizontal trigger mode by default. What's wrong with using vertical trigger mode?

In vertical trigger mode, listenfd notifies the application layer only when it changes from unread event state to read event state, and epoll will not notify the application layer until the application layer has finished processing all read events on the listenfd.

In other words, after the application layer receives the notification of read events on listenfd, it needs to process all read events on listenfd, and the next time there are read events on listenfd, it will notify the application layer.

Going back to the problem of accept, in vertical trigger mode, when epoll notifies the application layer that there are readable events on the listenfd, the application layer calls accept. Because the number of file descriptors opened by the process has reached the limit, the accept call fails.

That is, the readable events on the listenfd have not been handled, and until the application layer has finished processing the readable events on the listenfd, epoll will no longer notify the application layer that there are readable events on the listenfd.

If a new client connection arrives before the application layer finishes processing the readable event on the listenfd, the epoll will not notify the application layer that there is a readable event on the listenfd, which will lead to a serious problem: as long as the accept has the error code of EMFILE, it can no longer accept the client connection.

Therefore, when EMFILE appears, there will be problems whether using the horizontal trigger mode or the vertical trigger mode of epoll.

How to solve

EMFILE indicates that the number of file descriptors opened by the process has reached the limit. You can increase this value, but this will cure the symptoms rather than the root of the problem.

Originally, the system set the upper limit of the number of file descriptors in order to limit the excessive occupation of system resources by the process. Moreover, how appropriate the value is, it cannot be infinitely large, so the way to adjust the upper limit is not the most appropriate way.

When accept succeeds, a new file descriptor will be returned. If the number of file descriptors opened by the process has reached the limit, a failure will be returned.

If you can close an idle file descriptor at this time, give up a quota, and then call accept, it will be created successfully. The specific processing steps in this way are as follows:

1. Prepare an idle file descriptor idlefd in advance, which is equivalent to occupying a "pit" bit first.

2. Call close to close idlefd. After closing, the process will get a file descriptor quota.

3. Call the accept function again, and the new file descriptor clientfd will be returned. Call the close function immediately and close clientfd.

4. Recreate the free file descriptor idlefd, reoccupy the "pit" bit, and use it again when this happens.

Since the test code is relatively long, it will not be posted here. If you are interested, you can obtain it at the end of the article. Here is the pseudo code for dealing with EMFILE:

Int ret = accept (listenfd, (struct sockaddr*) & addr, sizeof (addr)); if (- 1 = = ret) {if (errno = = EMFILE) {/ / close the idle file descriptor and release the "pit" bit close (idlefd); / / accept clientfd clientfd = accept (listenfd, nullptr, nullptr); / / close clientfd to prevent the readable event close (clientfd) on listenfd from being triggered all the time / / reoccupy the "pit" bit idlefd =: open ("/ dev/null", O_RDONLY | O_CLOEXEC);}} Thank you for reading this article carefully. I hope the article "how to deal with Emfile problems in Accept" shared by the editor will be helpful to you. At the same time, I also hope you will support us, pay attention to the industry information channel, and more related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.