Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to reasonably set the maximum number of threads and core threads in the java thread pool. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Thread pool reasonably sets the maximum number of threads and the number of core threads

In the work, there is such a scenario that requires an algorithm to deal with tens of millions of levels of data, mostly adding, deleting and checking operations. At this point, you need to use multithreading to process.

At first, @ Configuration@EnableAsync (proxyTargetClass = true) / / enable asynchronous task support with @ EnableAsync annotation @ ComponentScan ({"com.ctfojt.auditbcarslogo.service"}) / / scan package public class ThreadPoolConfig implements AsyncConfigurer {@ Override public Executor getAsyncExecutor () {ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor (); taskExecutor.setCorePoolSize (10); / / Core thread size taskExecutor.setMaxPoolSize (20) must be added / / maximum thread size taskExecutor.setQueueCapacity (500); / / maximum queue capacity / / this parameter needs to be set when the number of tasks submitted is greater than QueueCapacity, but none of the tasks provided by spring does not satisfy the business scenario. You can customize one, or you can make sure that it does not exceed QueueCapacity taskExecutor.setRejectedExecutionHandler (new ThreadPoolExecutor.CallerRunsPolicy ()); taskExecutor.setWaitForTasksToCompleteOnShutdown (true); taskExecutor.setAwaitTerminationSeconds (10). TaskExecutor.setThreadNamePrefix ("BCarLogo-Thread-"); taskExecutor.initialize (); return taskExecutor;}}

This configuration is inefficient and can process more than 300,000 data a day. Later, as more and more data are inserted into the table, the processing speed slows down, and after a day or two of running, it can handle more than 100000. It can't meet the demand at all.

Later, query the configuration of thread pool cores on the Internet.

Most of them go like this:

Note: IO-intensive (practical experience of a large factory) core threads = CPU cores / (1-blocking coefficient) or CPU-intensive: core threads = CPU cores + 1 IO-intensive: core threads = CPU cores * 2

I also try to configure it in this way, but it turns out that the efficiency is not ideal and it doesn't improve much.

In the end, this is how I configured it.

As a result, the efficiency has been greatly improved, running tens of millions of levels of data in less than a day's data.

/ / get the number of cores of the current machine public static final int cpuNum = Runtime.getRuntime () .availableProcessors (); @ Override public Executor getAsyncExecutor () {ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor (); taskExecutor.setCorePoolSize (cpuNum); / / Core thread size taskExecutor.setMaxPoolSize (cpuNum * 2); / / maximum thread size taskExecutor.setQueueCapacity / / maximum queue capacity / / this parameter needs to be set when the number of tasks submitted is greater than QueueCapacity, but none of the tasks provided by spring can meet business scenarios. You can customize one, or you can taskExecutor.setRejectedExecutionHandler (new ThreadPoolExecutor.CallerRunsPolicy ()); taskExecutor.setWaitForTasksToCompleteOnShutdown (true); taskExecutor.setAwaitTerminationSeconds (60); taskExecutor.setThreadNamePrefix ("BCarLogo-Thread-"). TaskExecutor.initialize (); return taskExecutor;}

Solved the problem perfectly!

The difference between the number of core threads and the maximum number of threads in thread pool thread pool strategy

CorePoolSize: core threads; maximunPoolSize: maximum number of threads

Whenever there is a new task to the thread pool

Step 1: first determine whether the current number of threads in the thread pool has reached corePoolSize. If not, the new thread runs the task, and after the task ends, the thread is left in the thread pool without destruction. If the current number of threads has reached corePoolSize, proceed to the next step.

Step 2: determine whether the work queue (workQueue) is full, then submit the new task to the work queue, and move on to the next step

Step 3: determine whether the number of threads in the thread pool has reached maxumunPoolSize. If not, create a new worker thread to perform the task, and if so, use the saturation policy to handle the task. Note: when the number of threads in the thread pool exceeds the corePoolSize, the thread is terminated whenever the idle time of a thread exceeds the keepAliveTime. Until the number of threads in the thread pool is not greater than corePoolSize.

(as you can see from step 3, in general, corePoolSize threads remain in the Java thread pool for a long time. )

Saturation strategy

The strategy adopted when the work queue is full and the number of threads reaches maximunPoolSize

AbortPolicy: default policy; when a new task is submitted, an unchecked exception RejectedExecutionException is thrown directly, which can be caught by the caller.

CallerRunsPolicy: neither discard the task nor throw an exception, but run the new task using the caller's thread.

DiscardPolicy: discards the new task without throwing an exception.

DiscardOldestPolicy: call the poll method to discard the task of the queue leader, and then try to submit a new task

Custom policy: customized according to the needs of users.

This is the end of the article on "how to reasonably set the maximum number of threads and core threads in the java thread pool". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.