Shulou(Shulou.com)06/01 Report--

Today, I would like to share with you how to use asynchronous thread pool to achieve batch data push in production environment in SpringBoot. The content is detailed and the logic is clear. I believe most people still know too much about this knowledge, so share this article for your reference. I hope you can get something after reading this article.

Preface

SpringBoot uses asynchronous thread pools:

1. Write thread pool configuration class and customize a thread pool.

2. Define an asynchronous service

3. Use @ Async annotation to point to the defined thread pool

This is described by a case I used in my work. My company is in the medical industry, and sensitive data needs to be reported to a regulatory platform, so there is a scheduled task to report behavior when the traffic is small (usually in the early morning). However, in a special period, it is necessary to report data in large quantities during working hours, and it is required to be completed in a short time. At this time, consider writing a special asynchronous reporting interface for manual execution, using thread pool to report, which greatly improves the speed.

Write thread pool configuration class import lombok.extern.slf4j.Slf4j;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.scheduling.annotation.EnableAsync;import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import java.util.concurrent.Executor;import java.util.concurrent.ThreadPoolExecutor; / * class name: ExecutorConfig *

* Class description: thread pool configuration * * @ author guoj * @ date 2021-09-07 09:00 * / @ Configuration@EnableAsync@Slf4jpublic class ExecutorConfig {/ * define data reporting thread pool * @ return * / @ Bean ("dataCollectionExecutor") public Executor dataCollectionExecutor () {ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor () / / number of core threads: number of cores of the current machine executor.setCorePoolSize (Runtime.getRuntime (). AvailableProcessors ()); / / maximum number of threads executor.setMaxPoolSize (Runtime.getRuntime (). AvailableProcessors () * 2); / / queue size executor.setQueueCapacity (Integer.MAX_VALUE) / / Thread name prefix executor.setThreadNamePrefix ("sjsb-") in thread pool; / / reject policy: reject executor.setRejectedExecutionHandler (new ThreadPoolExecutor.AbortPolicy ()) directly; / / perform initialization executor.initialize (); return executor;}}

PS:

1) it is important to note that ThreadPoolTaskExecutor thread pool must be defined here, otherwise asynchronous comments of springboot will execute the default thread pool, which may lead to CPU soaring and memory overflow caused by thread blocking. You can refer to Ali's development manual for this, which is explicitly mentioned in the thread pool definition section.

2) define the thread pool name in the @ Bean annotation, which will be used later in the asynchronous annotation.

Write services for asynchronous services / * asynchronous methods without affecting the running of the main program. * / @ Servicepublic class AsyncService {private final Logger log = LoggerFactory.getLogger (AsyncService.class); / * send SMS * / @ Async ("sendMsgExecutor") public void sendMsg (String access_token, Consult item, Map configMap) {/ / here write SMS service / / 1, buildConsultData (); / / 2, sendMsg () } / * send Wechat subscription message * / @ Async public void sendSubscribeMsg (String access_token, Consult item, Map configMap) {/ / write and send Wechat subscription message service here / / 1, buildConsultData (); / / 2, sendSubscribeMsg () } / * data and report * / @ Async ("dataCollectionExecutor") public void buildAndPostData (String access_token, Consult item, Map configMap) {/ / write the reporting business here, such as stitching data, and then report. / / 1, buildConsultData (); / 2, postData ();}}

PS:

1) the above are code snippets. Personal experience believes that it is best to define an asynchronous service to store each asynchronous method, so as to avoid some misoperations during coding, such as the asynchronous method is not void or private modification, resulting in the invalidation of the @ Async annotation. At the same time, it is more flexible to arrange each annotation to point to a different custom thread pool.

2) the name in the @ Async annotation is the custom thread pool name defined above, so that the asynchronous thread is obtained from the specified thread pool when the business is executed.

Asynchronously batch report data @ Autowiredprivate AsyncService asyncService; / * * manually report consultation records in thread pool mode. * / public void manualUploadConsultRecordsAsync (String channel, Date startTime, Date endTime) {/ / query consultation records List consultList = consultService .findPaidListByChannelAndTime (channel, startTime, endTime, configMap.get ("serviceId"); if (! CollectionUtils.isEmpty (consultList)) {log.debug ("[SendWZDataService] [manualUploadConsultRecordsAsync] > manually report consultation records, a total of [{}]", consultList.size ()) ConsultList.forEach ((item)-> {try {/ / Asynchronous calls, using thread pools. AsyncService.buildAndPostData (access_token, item, configMap);} catch (Exception ex) {log.error ("[SendWZDataService] [manualUploadConsultRecordsAsync] > manually report an exception in the consultation record:", ex);}});}} this is all the content of the article "how to use asynchronous thread pool in SpringBoot to achieve batch data push in production environment". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.