Shulou(Shulou.com)06/03 Report--

Background

Concurrent functions are more or less used in our daily business development process. Then in the process of using the concurrency feature, we are sure to encounter the following problem

How big is the concurrent thread pool?

Programmers who are usually a little older may have heard the saying (where N stands for the number of CPU)

CPU-intensive applications, thread pool size set to N + 1IO-intensive applications, thread pool size set to 2N

Is this statement correct or not?

In fact, this is extremely incorrect. Then why?

First of all, from the negative point of view, assuming this statement is true, it doesn't matter how many services we deploy on a server. This is not true because the size of the thread pool is only related to the number of cores of the server. How exactly should the size be set? Suppose the application is a mixture of both, in which tasks are both CPU-intensive and IO-intensive, so how do we set them up? Can only throw the hard drive to decide?

So how exactly do we set the thread pool size? Are there any specific practical methods to guide people to land? Let's take a closer look at it.

Little's Law (Little's rule)

The number of requests in a system is equal to the product of the arrival rate of the request and the average time spent on each individual request

Assuming that the server is single-core, and the number of requests is guaranteed (QPS): 10, and it takes 1 second to process a request, then the server has 10 requests being processed at every moment, that is, 10 threads are needed.

Similarly, we can use the Little's law rule to determine the thread pool size. We only need to calculate the request arrival rate and the average request processing time. Then, put the above values into the Little's law to calculate the average number of requests in the system. The estimation formula is as follows

* Thread pool size = ((thread IO time + thread CPU time) / thread CPU time) number of CPU * *

Concrete practice

Through the formula, we know that three specific values are needed.

Time consumed by a request (thread IO time + thread CPU time) the request calculation time (thread CPU time) number of CPU requests elapsed time

In the Web service container, you can use Filter to intercept the time spent before and after obtaining the request.

Public class MoniterFilter implements Filter {private static final Logger logger = LoggerFactory.getLogger (MoniterFilter.class); @ Override public void doFilter (ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {long start = System.currentTimeMillis (); HttpServletRequest httpRequest = (HttpServletRequest) request; HttpServletResponse httpResponse = (HttpServletResponse) response; String uri = httpRequest.getRequestURI (); String params = getQueryString (httpRequest) Try {chain.doFilter (httpRequest, httpResponse);} finally {long cost = System.currentTimeMillis ()-start; logger.info ("access url [{} {}], cost time [{}] ms)", uri, params, cost);} private String getQueryString (HttpServletRequest req) {StringBuilder buffer = new StringBuilder ("?"); Enumeration emParams = req.getParameterNames () Try {while (emParams.hasMoreElements ()) {String sParam = emParams.nextElement (); String sValues = req.getParameter (sParam); buffer.append (sParam) .append ("=") .append (sValues) .append ("&");} return buffer.substring (0, buffer.length ()-1) } catch (Exception e) {logger.error ("get post arguments error", buffer.toString ());} return ";}} CPU calculation time

CPU calculation time = total request time-CPU IO time

Suppose the request has an operation to query DB, as long as you know the CPU IO time of the query DB, the calculation time will come out. Let's take a look at how to keep it simple and clearly record the DB query time. It takes time to get thread IO by adding AOP sections through (JDK dynamic proxy / CGLIB). The code is as follows, please see

Public class DaoInterceptor implements MethodInterceptor {private static final Logger logger = LoggerFactory.getLogger (DaoInterceptor.class); @ Override public Object invoke (MethodInvocation invocation) throws Throwable {StopWatch watch = new StopWatch (); watch.start (); Object result = null; Throwable t = null; try {result = invocation.proceed ();} catch (Throwable e) {t = e = = null? Null: e.getCause (); throw e;} finally {watch.stop (); logger.info ("({} ms)", watch.getTotalTimeMillis ());} return result;}} CPU number

The number of logical CPU, the number of CPU referenced when setting the thread pool size

Cat / proc/cpuinfo | grep "processor" | wc-l summary

It is not easy to configure the thread pool size properly, but through the above formula and specific code, we can quickly and quickly calculate the size of the thread pool. In the end, however, we still need to fine-tune through the stress test, and only through the test of the stress test can we finally ensure that the configuration size is accurate.

Referenc

Little's law

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.