Shulou(Shulou.com)06/03 Report--

This article introduces the knowledge of "how to understand Tomcat highly concurrent connection pool and thread pool". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

one

The entry component of Tomcat that handles user requests is called Connector, which has two main implementations: BIO (blocking io) and NIO (non-blocking io).

To put it simply, the implementation of BIO is an improvement on the above multi-threaded version, the main point is to change "one thread processing for each connection" to "each connection is submitted to the thread pool for processing". Although the working behavior of thread pools varies according to different configurations, generally speaking, the principle of using thread pools is that a large number of tasks can be executed by creating a small number of threads, and since at most a fixed number of threads are executed at the same time, the remaining tasks will be buffered in queue. From this point of view, this is a typical producer-consumer model. Back to the tomcat BIO,acceptor, the connection is continuously received and then submitted to the thread pool for execution, and the acceptor is the producer; each thread in the thread pool is the consumer and is responsible for processing the request.

Because the socket connection is a persistent connection, the creation and destruction of the connection is also resource-consuming, so a keep-alive header is added to the http protocol, which means to remind the server that after returning http response, do not disconnect the socket and continue to process subsequent http requests. The purpose of doing this is to improve the reusability of resources. So, what is the problem with the implementation of tomcat BIO in the keep-alive scenario? If a threaded socket needs to keep keep-alive, it needs to block there after executing a http request to wait for the next http request and cannot end immediately (until timeout); in some cases, there may be a large number of blocked threads and new connections cannot be processed.

Based on this, NIO can solve this problem. The implementation of NIO and BIO in the request processing part is consistent and is based on thread pool. The difference is that NIO's acceptor implementation is based on jdk nio. After receiving a connection, it registers socketChannel with poller's selector, and when socketChannel has data to read, poller submits the connection to thread pool for processing. Going back to the keep-alive scenario above, when a thread has finished processing a http request, it can end immediately, and the current connection goes back to selector to listen for the next http request. As a result, NIO-based threads of execution do not experience BIO-based blocking.

The core of NIO is that selector,selector can recognize connections that are already ready and connections that do not have ready; as mentioned in a previous multithreading article (comparing Java with .NET multithreaded programming), jdk's concurrency API has a CompletionService class, which is somewhat similar to the principle of nio.

Because of the inherent advantages of NIO, tomcat has set NIO as the default Connector since version 8.0, while BIO has been removed directly from version 8.5.

two

On tomcat's website, there is a description of how to handle requests with high concurrency:

Each incoming request requires a thread for the duration of that request. If more simultaneous requests are received than can be handled by the currently available request processing threads, additional threads will be created up to the configured maximum (the value of the maxThreads attribute). If still more simultaneous requests are received, they are stacked up inside the server socket created by the Connector, up to the configured maximum (the value of the acceptCount attribute). Any further simultaneous requests will receive "connection refused" errors, until resources are available to process them.

-https://tomcat.apache.org/tomcat-7.0-doc/config/http.html

Personally, I think it does not reflect the effect of the parameter maxConnections, so it should be: if maxConnections is less than maxThreads, the maximum number of threads created is the value of maxConnections, and the maximum number of connections is also the value of maxConnections; but if maxConnections is greater than maxThreads, the maximum number of threads created is the value of maxThreads, and the maximum number of connections is the value of maxConnections.

Because of the difference between the underlying implementation of BIO and NIO, the value of configuring maxConnections also needs to be considered, which is reflected in the default value of maxConnections (the default value is maxThreads for BIO,maxConnections and 10000 for NIO,maxConnections):

three

As mentioned above, the process of receiving and processing requests in tomcat is actually a producer-consumer model. Configurations that affect the high concurrency of tomcat can also be considered from these two aspects first:

Producer

Consumer

Queue

four

A brief summary:

The essence of thread pools is to save the overhead of constantly creating and destroying threads; coupled with the use of queue, an additional layer of buffer is added, which relieves the pressure on the computer to some extent. Of course, the configuration of the thread pool needs to be carefully considered according to the tasks to be handled (CPU-intensive or io-intensive).

The biggest difference between BIO and NIO in Tomcat is whether blocking is required when reading the next request, which is especially important for keep-alive scenarios. NIO can greatly improve throughput.

The generator-consumer model based on queue is also often used at the system architecture level to buffer the gap of processing speed between producers and consumers, such as the second kill system.

This is the end of the content of "how to understand Tomcat highly concurrent connection pool and thread pool". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.