Shulou(Shulou.com)05/31 Report--

This article introduces the knowledge of "what is the number of connections and thread pool in Tomcat". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

One: preface

When using tomcat, you often encounter configuration problems such as the number of connections and threads. To really understand these concepts, you must first understand the Connector of Tomcat.

In the Tomcat configuration file server.xml: the main function of Connector is to receive connection requests, create Request and Response objects to exchange data with the requester, then assign threads to Engine (that is, the Servlet container) to process the request, and pass the resulting Request and Response objects to Engine. When Engine finishes processing the request, it also returns the response to the client through Connector.

It can be said that the Servlet container processes requests, which needs to be scheduled and controlled by Connector. Connector is the backbone of Tomcat processing requests, so the configuration and use of Connector have an important impact on the performance of Tomcat.

According to the different protocols, Connector can be divided into HTTP Connector, AJP Connector and so on. This paper only discusses HTTP Connector.

Two: Nio, Bio, APR

Protocol of 1.Connector

Connector uses a different protocol when processing HTTP requests. Different versions of Tomcat support different protocol, and the most typical protocol includes BIO, NIO, and APR (these three are supported in Tomcat7, Tomcat8 adds support for NIO2, and when it comes to Tomcat8.5 and Tomcat9.0, BIO support is removed).

BIO is Blocking IO, and as its name implies, a blocking IO;NIO is a Non-blocking IO, then a non-blocking IO. APR is Apache Portable Runtime and Apache portable runtime, which can achieve high scalability and high performance by using local libraries. Apr is the preferred mode for running highly concurrent applications on Tomcat, but packages such as apr, apr-utils and tomcat-native need to be installed.

two。 How to specify protocol

Which protocol is used by Connector can be specified by the protocol attribute in the element, or the default value can be used.

The specified protocol value and the corresponding protocol are as follows:

HTTP/1.1: default. The protocol used is related to the version of Tomcat.

Org.apache.coyote.http11.Http11Protocol:BIO

Org.apache.coyote.http11.Http11NioProtocol:NIO

Org.apache.coyote.http11.Http11Nio2Protocol:NIO2

Org.apache.coyote.http11.Http11AprProtocol:APR

If protocol is not specified, the default value HTTP/1.1 is used, which means as follows: in Tomcat7, automatically select to use BIO or APR (if you find the local library needed by APR, use APR, otherwise use BIO); in Tomcat8, automatically select to use NIO or APR (if you find the local library needed by APR, use APR, otherwise use NIO).

How is 3.BIO/NIO different?

Whether it's BIO or NIO,Connector, the process for handling requests is roughly the same:

Receive the connection in the accept queue (when the client sends a request to the server, if the client completes the three-way handshake to establish a connection with the OS, the OS places the connection into the accept queue); gets the requested data in the connection, generates request; calls the servlet container to process the request, and returns response. To facilitate the following explanation, first clarify the relationship between connection and request: the connection is at the TCP level (transport layer), and the corresponding socket; request is at the HTTP level (application layer), which must rely on the connection implementation of TCP; multiple HTTP requests may be transmitted in a TCP connection.

In Connector implemented by BIO, the main entity that processes the request is the JIoEndpoint object. JIoEndpoint maintains Acceptor and Worker:Acceptor to receive socket, and then finds free threads from the worker thread pool to process socket, and if there are no free threads in the worker thread pool, Acceptor will block. Where Worker is the thread pool that comes with Tomcat, and the principle is similar to Worker if other thread pools are configured.

In Connector implemented by NIO, the main entity that processes the request is the NIoEndpoint object. In addition to Acceptor and Worker, Poller is also used in NIoEndpoint, as shown in the following figure (image source: http://gearever.iteye.com/blog/1844203)).

After Acceptor receives the socket, instead of directly using the thread in Worker to process the request, it first sends the request to Poller, and Poller is the key to the implementation of NIO. Acceptor sends requests to Poller through queues, using a typical producer-consumer model. In Poller, a Selector object is maintained; when the Poller fetches the socket from the queue, it registers with the Selector; then, by traversing the Selector, it finds the readable socket in it, and uses threads in the Worker to process the corresponding request. Like BIO, Worker can be replaced by a custom thread pool.

From the above process, we can see that in the process of NIoEndpoint processing the request, whether the Acceptor receives the socket or the thread processes the request, the blocking method is still used; but in the process of "reading the socket and giving it to the thread in the Worker", the non-blocking NIO implementation is used, which is the most important difference between the NIO mode and the BIO mode (other differences have little impact on performance, so forget for the time being). This difference can lead to a significant improvement in Tomcat efficiency in the case of large concurrency:

Currently, most HTTP requests use persistent connection (the default keep-alive for HTTP/1.1 is true). Persistent connection means that after the end of the current request, if no new request arrives, the socket of a TCP will not be released immediately, but will not be released until timeout. If you use BIO, the process of "reading the socket and handing it to the thread in the Worker" is blocked, which means that while the socket waits for the next request or waits for release, the worker thread dealing with the socket will always be occupied and cannot be released; therefore, the number of socket that Tomcat can process at the same time cannot exceed the maximum number of threads, and performance is greatly limited. With NIO, the process of "reading socket and giving it to threads in Worker" is non-blocking, and when socket is waiting for the next request or waiting for release, it does not occupy the worker thread, so the number of socket that Tomcat can handle at the same time is much greater than the maximum number of threads, and concurrent performance is greatly improved.

Three: acceptCount, maxConnections, maxThreads

Review the process of Tomcat processing the request: receive the connection in the accept queue (when the client sends the request to the server, if the client completes a three-way handshake with OS to establish a connection, OS puts the connection into the accept queue); gets the requested data in the connection, generates request; calls the servlet container to process the request, and returns response.

Accordingly, the functions of several parameters in Connector are as follows:

1.acceptCount

The length of the accept queue; when the number of connections in the accept queue reaches acceptCount, the queue is full and all incoming requests are rejected. The default value is 100.

2.maxConnections

The maximum number of connections received and processed by Tomcat at any time. When the number of connections received by Tomcat reaches maxConnections, the Acceptor thread will not read the connections in the accept queue; the threads in the accept queue will block until the number of connections received by Tomcat is less than maxConnections. If set to-1, the number of connections is unlimited.

The default value is related to the protocol used by the connector: the default value for NIO is 10000, and the default value for BIO is 8192, while the default value for BIO is maxThreads (if Executor is configured, the default value is maxThreads for Executor).

Under windows, the maxConnections value of APR/native is automatically adjusted to an integral multiple of the maximum of 1024 below the setting value; if set to 2000, the maximum value is actually 1024.

3.maxThreads

The maximum number of request processing threads. The default value is 200 (both Tomcat7 and 8). If the Connector is bound to Executor, this value is ignored because the Connector will use the bound Executor instead of the built-in thread pool to execute the task.

MaxThreads specifies the maximum number of threads, not the actual number of CPU for running; in fact, the size of maxThreads is much larger than the number of CPU cores. This is because the thread processing the request may spend very little time on computing and may be blocking most of the time, such as waiting for the database to return data, waiting for the hard disk to read and write data, and so on. Therefore, at some point, only a small number of threads are actually using physical CPU, and most threads are waiting; therefore, it is reasonable that the number of threads is much greater than the number of physical cores.

In other words, Tomcat can keep CPU busy and greatly improve CPU utilization by using much more threads than the number of CPU cores.

4. Parameter setting

(1) the setting of maxThreads is not only related to the characteristics of the application, but also to the number of CPU cores of the server. As you can see from the previous introduction, the number of maxThreads should be much larger than the number of CPU cores; and the larger the number of CPU cores, the larger the maxThreads; the less dense the CPU in the application (the denser the IO), the larger the maxThreads should be, so that you can make full use of CPU. Of course, the larger the value of maxThreads, the better. If the maxThreads is too large, then CPU will spend a lot of time on thread switching, and the overall efficiency will be reduced.

(2) the setting of maxConnections is related to the running mode of Tomcat. If tomcat uses BIO, the value of maxConnections should be the same as that of maxThreads; if tomcat uses NIO, then, similar to the default value of Tomcat, the maxConnections value should be much larger than maxThreads.

(3) from the previous introduction, we can know that although the number of connections that tomcat can handle at the same time is maxConnections, the number of connections that can be received in the server at the same time is maxConnections+acceptCount. The setting of acceptCount has something to do with how the application expects to react if the connection is too high. If the setting is too large, the waiting time for the incoming request will be very long; if the setting is too small, the subsequent request for entry will immediately return connection refused.

Four: thread pool Executor

The Executor element represents a thread pool in Tomcat that can be shared by other components; to use this thread pool, the component needs to specify the thread pool through the executor attribute.

Executor is an embedded element of the Service element. Generally speaking, it is the Connector component that uses the thread pool; in order for Connector to use the thread pool, the Executor element should be placed before the Connector. Examples of Executor and Connector configurations are as follows:

And

The main properties of Executor include:

Name: the tag for this thread pool

MaxThreads: the maximum number of active threads in the thread pool. The default is 200 (Tomcat7 and 8 are both).

MinSpareThreads: the minimum number of threads held in the thread pool, which is 25

MaxIdleTime: the maximum amount of time a thread is idle and shuts down when idle exceeds this value (unless the number of threads is less than minSpareThreads), in ms. Default is 60000 (1 minute).

Daemon: whether it is a background thread. Default is true.

ThreadPriority: thread priority, default is 5

NamePrefix: the prefix of the thread name. The thread name in the thread pool is: namePrefix+ thread number.

Five: query the current status

The concepts of the number of connections and threads in Tomcat and how to set them are described above, and how to view the number of connections and threads in the server is shown below.

To view the status of the server, it is roughly divided into two scenarios: (1) using off-the-shelf tools, and (2) directly using Linux commands to view.

Off-the-shelf tools, such as JDK's jconsole tool, can easily view thread information (in addition, you can also view CPU, memory, classes, JVM basic information, etc.), Tomcat's own manager, charging tool New Relic and so on. The following figure shows the interface for jconsole to view thread information:

Here's how to check the number of connections and threads in the server through the Linux command line.

1. Number of connections

Assuming that the port on which Tomcat receives http requests is 8083, you can use the following statement to view the connection:

Netstat-nat | grep 8083

The results are as follows:

As you can see, one connection is in the listen state, listening for requests; in addition, there are four established connections (ESTABLISHED) and two connections waiting to be closed (CLOSE_WAIT).

two。 Thread

The ps command can view the status of the process, such as executing the following command:

Ps-e | grep java

The result is as follows:

As you can see, only one process's information is printed; 27989 is the thread id,java is the java command that is executed. This is because you start a tomcat and all the internal work is done in this process, including the main thread, garbage collection thread, acceptor thread, request processing thread, and so on.

You can see how many threads there are in the process with the following command; where nlwp means number of light-weight process.

Ps-o nlwp 27989

As you can see, there are 73 threads inside the process; however, 73 does not exclude threads in the idle state. To get the number of threads that are actually in running, you can do this with the following statement:

Ps-eLo pid, stat | grep 27989 | grep running | wc-l

Among them, ps-eLo pid, stat can find all threads and print the process number and the current state of the thread; two grep commands filter the process number and thread status respectively; wc counts. The output result of ps-eLo pid, stat | grep 27989 is as follows:

Only part of the result is captured in the figure; Sl indicates that most threads are idle.

This is the end of the content of "what is the number of connections and thread pool in Tomcat". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.