Shulou(Shulou.com)06/01 Report--

How to understand the server single I/O thread + worker thread pool model architecture and implementation points, I believe that many inexperienced people are helpless about this, this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

We know that server concurrency models can usually be divided into single-threaded and multi-threaded models, where threads usually refer to "I/O threads," that is,"management threads" responsible for I/O operations and coordinating the allocation of tasks, while actual requests and tasks are usually handled by so-called "worker threads." In a multithreaded model, each thread is both an I/O thread and a worker thread. So what we're talking about here is the single I/O thread + multiworker thread model, which is also the most common server concurrency model. This model is ubiquitous in the server code of my project. It is also known as the "semi-synchronous/semi-asynchronous" model, and it is also a manifestation of the producer/consumer (especially multi-consumer) model.

This architecture is mainly based on the idea of I/O multiplexing (mainly epoll, select/poll is outdated), through single-threaded I/O multiplexing, can achieve efficient concurrency, while avoiding multi-threaded I/O switching back and forth of various overhead, clear thinking, easy to manage, and based on thread pool multi-worker threads, but also can give full play to and take advantage of multi-threading, using thread pool, further improve resource reuse and avoid excessive threads.

1. model architecture

2 Key points of realization

2.1 Single I/O thread epoll

Implementing the epoll model with a single I/O thread is the first technical point of this architecture. The main ideas are as follows:

A single thread creates an epoll and waits for an I/O socket to arrive, adds it to the epoll and takes an idle worker thread from the thread pool, handing over the actual task to the worker thread.

Pseudocode:

Pseudocode may not be written very well, in fact, is the basic epoll use.

But pay attention to the use of thread pool, if the thread pool can not get idle worker thread, still need to do some processing.

2.2 Thread pool implementation points

When server starts, create a certain number of worker threads to join the thread pool, such as (20), for I/O threads to access;

Whenever an I/O thread requests an idle worker thread, an idle worker thread is taken from the pool to process the corresponding request;

When the request is processed and the corresponding I/O connection is closed, the corresponding thread is recycled and put back into the thread pool for next use;

If there are no free worker threads when requesting the free worker thread pool, the following can be done:

(1)If the total number of "managed" threads in the pool does not exceed the maximum allowed value, a new batch of worker threads can be created to join the pool and one of them can be returned for use by the I/O thread;

(2)If the total number of Managed threads in the pool has reached a maximum and new threads should not be created, wait a short time and try again. Note that because I/O threads are single-threaded and should not be blocked waiting here, the management of the thread pool should be done by a dedicated management thread, including creating new worker threads. At this point, the management thread blocks waiting (e.g., using conditional variables and waiting to wake up), and after a short time, there should be free worker threads available in the thread pool. Otherwise, there is a problem with server load estimation.

After reading the above content, do you know how to understand the server single I/O thread + worker thread pool model architecture and implementation points? If you still want to learn more skills or want to know more related content, welcome to pay attention to the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.