Shulou(Shulou.com)06/02 Report--

Finally began to learn epoll, although there are still many things I do not understand, but from theory to practice, I believe that after writing a specific framework, everything will be much clearer.

1. First of all, a memory pool is required to:

Reduce frequent allocation and release, improve performance while avoiding the problem of memory fragmentation

Be able to store longer data, don't be foolish enough to pre-allocate only a maximum length

It is a good idea to implement memory pool based on SLAB algorithm: allocate multiple blocks of different sizes and return the smallest block larger than the length of the request when requesting. For containers, it is quite easy to deal with the allocation and recovery of fixed blocks. Of course, remember that it needs to be designed to be thread-safe, spin locks are better, and read-write spin locks are even better.

The growth management of allocated content is a problem, such as the need for 1KB space for the first time, and the need for 4KB space for the second time as data continues to be written. The expansion of space is easy to implement, but it must involve data copy. Even, the demand for expansion is very large, hundreds of megabytes of data, so it is not easy to do. There is no better idea at the moment, like STL, exponential growth allocation strategy, although copying data is inevitable, but at least the probability of redistribution is getting smaller and smaller.

As mentioned above, if the need for hundreds of megabytes of data expansion, using memory-mapped files to manage is a good way: after mapping files, although it takes up a lot of virtual memory, physical memory is allocated only when writing, and madvice () plus sequential write optimization recommendations, physical memory consumption will also be reduced.

It is not wise to use string or vector to manage memory, although it is simple, but STL is not suitable for server software development, especially when stability and performance requirements are high.

2. The second thing to consider is object pooling, similar to memory pooling:

Reduce the allocation and release of objects. In fact, the C++ object is also struct, so it is not difficult to manually initialize and clean up the construction and destructions, and to maintain the recycling of the same buffer.

Can be designed as an object pool can only store one kind of object, then the implementation of the object pool is actually the pooling management of fixed memory blocks, which is very simple. After all, the number of objects is very limited.

3. The third thing you need is the queue:

If you can anticipate the limit of processing capacity, it would be nice to use a fixed-size ring queue as a buffer. One producer and one consumer is a common application scenario. Ring queue has its classic lock-independent algorithm. In the scenario of one thread reading and one thread writing, the implementation is simple, the performance is high, and it does not involve the allocation and release of resources. Yes, it's really good!

When it comes to multiple producers and consumers, tbb::concurent_queue is a good choice, thread-safe or concurrency, but you don't know if the allocation and release of resources is well managed.

4. The fourth thing you need is a mapping table, or hash table:

Because epoll is triggered by events, and a series of processes may be scattered among multiple events, an intermediate state must be retained so that when the next event is triggered, it can be processed at the same location as the last one. To keep it simple, STL's hash_map is OK, but you have to deal with the lock yourself, which is troublesome to use in a multithreaded environment.

The best hash table in a multithreaded environment is tbb::concurent_hash_map.

5. The core thread is the event thread:

The event thread is the thread that calls epoll_wait () to wait for the event. In the example code, when a thread does everything and needs to develop a high-performance server, the event thread should focus on the processing of the event itself, put the socket handle that triggers the event into the corresponding processing queue, and the specific processing thread is responsible for the specific work.

6. Accept () a single thread:

The socket handle on the server (that is, the one that calls bind () and listen ()) is better to do accept () in a separate thread, blocking or non-blocking does not matter, compared with the communication of the entire server, the user access action is only a small part. Moreover, accept () is not placed in the loop of the event thread, reducing judgment.

7. A single receiving thread:

The receiving thread fetches the socket handle from the queue where the EPOLLIN event occurred, and then calls recv on that handle to receive data until there is no data in the buffer. The received data is written to the hash table with socket as the key, and there is a self-growing buffer in the hash table, which stores the data sent by the client.

This method is suitable for applications where the data sent by the client is very small, such as a HTTP server. If it is a file upload server, the receiving thread will always process a large amount of data from a connection, and the data processing of other clients will lead to hunger. Therefore, if it is a scenario such as a file upload server, it cannot be designed in this way.

8. A single sending thread:

The sending thread gets the SOCKET handles that need to send data from the send queue, and calls send () on these handles to send the data to the client. The queue means that the SOCKET handle is saved, and the specific information needs to be found in the hash table through the socket handle to locate the specific object. As mentioned above, the object of the client message has not only a variable length receive data buffer, but also a variable length send data buffer. When sending data, the specific worker thread does not directly call the send () function, but writes the data to the sending data buffer, and then places the SOCKET handle on the sending thread queue.

Another situation when a SOCKET handle is placed in the sending thread queue is that an EPOLLOUT event occurs in the event thread, indicating that there is free space in the TCP send buffer. At this time, you can put the SOCKET handle on the sending thread queue and trigger the call to send ().

It is important to note that when the sending thread sends a large amount of data, when send () is called frequently until the send buffer of TCP is full, it can no longer be sent. At this time, if you wait in a loop, the sending work of other users will be affected; if you do not continue to send, the ET mode of EPOLL may no longer generate events. The solution to this problem is to establish a queue within the sending thread, or to set a flag on the user information object and wait until the thread is idle before continuing to send the unfinished data.

9. A timer thread is required:

A master who uses epoll said: "it is almost impossible to rely on epoll to manage descriptors without divulging. The complete solution is to set a timeout for each fd. If the time exceeds the timeout, the fd will be dropped if it is not active."

Therefore, the timer thread periodically rotates the entire hash table to check whether the socket is inactive within a specified period of time. The inactive SOCKET considers it to be a timeout, and then the server actively closes the handle and reclaims resources.

10. Multiple worker threads:

The worker thread is triggered by the receiving thread: each time the receiving thread receives the data, it puts the SOCKET handle with the data into a work queue; the worker thread gets the SOCKET handle from the work queue, queries the hash table, locates to the user information object, and processes the business logic.

If the worker thread needs to send data, it first writes the data to the send buffer of the user information object, and then puts the SOCKET handle into the queue of the sending thread.

For task queues, receiving threads are producers and multiple worker threads are consumers; for sending thread queues, multiple worker threads are producers and sending threads are consumers. You need to pay attention to the lock here, and if you use tbb::concurrent_queue, it will be a lot easier.

11. Just using the scoket handle as the key of the hash table is not enough:

Suppose that the event thread has just put a SOCKET in the receive queue due to the occurrence of an EPOLLIN event, but then the client is unexpectedly disconnected, and the event thread deletes this item in the hash table because of the EPOLLERR event. Assuming that the receiving queue is long and the SOCKET where the exception occurred is still in the queue, when the receiving thread processes the SOCKET, it will not be able to index objects in the hash table through the SOCKET handle.

The difficulty is that the SOCKET handle is immediately used by another client, and the access thread creates an object in the hash table for this SCOKET. At this point, the two SOCKET with the same handle are actually two different clients. In extreme cases, this can happen.

The solution is to use socket fd + sequence as the key of the hash table, and the sequence is obtained by the access thread accumulating an integer value after each accept (). In this way, even if the SOCKET handle is reused, there will be no problem.

12. Monitoring, you need to consider:

The most problematic part of the framework is the worker thread: the processing speed of the worker thread is too slow, which will cause the queues to soar and eventually cause the server to crash. Therefore, you must limit the maximum size allowed for each queue, and you need to monitor the processing time of each worker thread, after which you should use some way to end the worker thread.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.