Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "what are the implementation methods of timing tasks". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

one。 Scheduled task crontab included with Linux

I don't know if you have ever encountered such a scenario: sometimes you need temporary statistical data on the line and then export it to an excel table. This requirement is sometimes more complex, just writing sql statements can not meet the requirements, which requires writing java code. Then the program is packed into a jar package and executed in an online environment. Finally, the generated excel file is downloaded to the local environment.

In order to reduce the impact on the online environment, we usually choose to execute the statistical program at 1: 00 to 2: 00 in the morning when the number of users is small. (in fact, around 4: 00 in the morning, the number of users is the least.)

As it is too late, there is no need for us to wait there for the results of the implementation. A scheduled task can be done.

So, which scheduled task is more appropriate for this situation?

The answer is: crontab of the linux system. (however, it does not rule out that some projects are not deployed in the linux system.)

To run crontab-e, you can edit the timer and then add the following command:

0 2 * / usr/local/java/jdk1.8/bin/java-jar / data/app/tool.jar > / logs/tool.log &

You can execute the tool.jar program regularly at 2: 00 a. M. every day, and output the log to the tool.log file. Of course, you can also write the following commands to execute java programs into shell scripts, which are more convenient to maintain.

Using this kind of timing task support, it is convenient to modify timing rules, and there is an interface to uniformly manage various timing scripts configured.

The basic format of the crontab command is as follows:

Crontab [parameter] [file name]

If no filename is specified, the command entered on the keyboard is received and loaded into crontab.

The comparison table of parameters and functions is as follows:

For the above parameters, if you do not specify a user with-u, the current user used by default.

Edit the contents of the file through the crontab-e command, with the following syntax:

[minutes] [hours] [date] [month] [week] specific tasks

Where:

Points, indicating how many minutes, range: 0-59

Hours, indicating how many hours, range: 0-23

Date, indicating the specific date, range: 1-31

Month, indicating how many months, range: 1-12

Week, indicating how many weeks, range: 0-7, both 0 and 7 represent Sundays

There are some special characters, such as:

* represents the time, for example: * 1 minutes * means that it is executed at 1: 00 a.m. Every day.

/ represents how often it is executed, for example: * / 5 * means that it is executed every 5 minutes.

Representatives support more than one, for example: 10 7, 9, 12 * * means that it is executed at 7, 9, and 12:10 every day.

-represents support a range, for example: 10 7-9 * * means that it is executed at 7, 8, and 09:10 every day.

In addition, by the way, crontab requires crond service support. Crond is a daemon used under linux to perform certain tasks periodically. After installing the linux operating system, crond service tools are installed by default, and crond services are self-starting by default. The crond process periodically checks every minute to see if there is a task to be executed, and if so, it automatically executes the task.

You can operate related services with the following command:

Service crond status / / View running status service crond start / / start the service service crond stop / / close the service service crond restart / / restart the service service crond reload / / reload the configuration

Advantages and disadvantages of using crontab:

Advantages: it is convenient to modify timing rules, support some complex timing rules, and manage all kinds of timing scripts through files.

Disadvantages: if there are too many scheduled tasks, it is difficult to find, and the operating system must be required to be linux, otherwise it cannot be executed.

two。 Scheduled task 1.Thread included with jdk

Dear friends, you read it correctly, the Thread class can really do scheduled tasks. If you look at the source code of some timed task frameworks, you will eventually find that their underlying Thread classes are also used.

The specific code to implement this scheduled task is as follows:

Public static void init () {new Thread (()-> {while (true) {try {System.out.println ("doSameThing"); Thread.sleep (1000 * 60 * 5);} catch (Exception e) {log.error (e);}). Start ();}

You can use the Thread class to do the simplest scheduled tasks, and there is an endless loop of while in the run method (and of course there are other ways) to perform our own tasks. One thing to pay special attention to is that you need to catch an exception with try...catch, otherwise, if an exception occurs, you will exit the loop directly, and you will not be able to continue execution next time.

The scheduled tasks done in this way can only be executed periodically and cannot be executed at a certain point in time.

In addition, the thread can be defined as a daemon thread, which can be executed silently in the background.

Usage scenarios: for example, projects sometimes need to download a file every 10 minutes, or read template files to generate static html pages every 5 minutes, etc., some simple periodic task scenarios.

Pros and cons of using the Thread class:

Advantages: this kind of scheduled task is very simple, low learning cost, easy to start, for those simple periodic tasks, it is a good choice.

Disadvantages: do not support the execution of tasks at a certain point in time, do not support delayed execution and other operations, the function is too simple to deal with some more complex scenarios.

2.TimerTimer

Class is a timer tool specially provided by jdk, which is used to plan and execute specified tasks in background threads. Under the java.util package, it should be used in conjunction with TimerTask.

The Timer class is actually a task scheduler that contains a timer thread that infinitely loops to get TimerTask from TaskQueue (this class implements the Runnable interface) and calls its run method to execute scheduled tasks asynchronously. We need to inherit the TimerTask class, implement its run method, and add our own business logic to that method.

The specific code to implement this scheduled task is as follows:

Public class TimerTest {public static void main (String [] args) {Timer timer = new Timer (); timer.schedule (new TimerTask () {@ Override public void run () {System.out.println ("doSomething");}, 2000 Magi 1000);}}

First instantiate a Timer class, then call its schedule method, where the TimerTask class is instantiated, and the business logic is written in the run method. The last two parameters of the schedule method are delay time and interval time, in milliseconds. In the above example, the scheduled task is set to be executed every 1 second, with a delay of 2 seconds.

There are mainly six methods:

Schedule (TimerTask task, Date time), which specifies that the task task executes at the specified time time

Schedule (TimerTask task, long delay), which specifies that the task task executes after the specified delay delay

Schedule (TimerTask task, Date firstTime,long period), which specifies that the task task will repeat the execution of peroid with a fixed delay frequency after the execution of the specified time firstTime.

Schedule (TimerTask task, long delay, long period), specify task task to repeat execution of peroid with fixed delay frequency after specifying delay delay

ScheduleAtFixedRate (TimerTask task,Date firstTime,long period), which specifies that the task task will repeat the execution of peroid with a fixed delay frequency after the execution of the specified time firstTime.

ScheduleAtFixedRate (TimerTask task, long delay, long period), specify task task to repeat execution of peroid with fixed delay frequency after specifying delay delay

However, using Timer to implement scheduled tasks has the following problems:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Because Timer executes tasks in a single thread, if one of the tasks takes a very long time, it will affect the execution of other tasks.

If TimerTask throws RuntimeException,Timer, it stops all tasks from running.

Pros and cons of using the Timer class:

Advantages: it is very convenient to implement multiple periodic scheduled tasks, and supports delayed execution, but also supports support after a specified time, and the function is still powerful.

Cons: if one of the tasks takes a very long time, it will affect the execution of other tasks. And if TimerTask throws RuntimeException,Timer, it will stop all tasks, so it is not recommended in Alibaba developer specification.

3.ScheduledExecutorService

ScheduledExecutorService is a scheduled task introduced in the JDK1.5+ version, which is located under the java.util.concurrent concurrent package.

ScheduledExecutorService is based on multithreading and is designed to solve the problem that multiple tasks will influence each other when Timer executes with a single thread.

It mainly consists of four methods:

Schedule (Runnable command,long delay,TimeUnit unit), a schedule with delay time, is executed only once, after which it can be blocked by Future.get () until the task is finished.

Schedule (Callable

Callable,long delay,TimeUnit unit), the scheduling with delay time is executed only once, after which the task can be blocked by Future.get () until the task is finished, and the execution result can be obtained.

ScheduleAtFixedRate, a task that executes at a fixed frequency. If the current task takes more time and exceeds the scheduled period period, it will be executed immediately after the end of the current task.

ScheduleWithFixedDelay, which means that the task is executed with a fixed delay, which calculates the start time relative to the end of the current task.

The specific code to implement this scheduled task is as follows:

Public class ScheduleExecutorTest {public static void main (String [] args) {ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool (5); scheduledExecutorService.scheduleAtFixedRate (()-> {System.out.println ("doSomething");}, 1000 Mae 1000, TimeUnit.MILLISECONDS);}}

Call the scheduleAtFixedRate method of the ScheduledExecutorService class to implement periodic tasks that are executed every 1 second, with a delay of 1 second each time.

This kind of scheduled task is used to replace the Timer class in Alibaba developer specification, and it is a good choice for multithreading to execute periodic tasks.

Advantages and disadvantages of ScheduledExecutorService:

Advantages: based on multi-threaded scheduled tasks, there is no correlation between multiple tasks, support periodic execution of tasks, and with delay function.

Disadvantages: some complex timing rules are not supported.

three。 Scheduled tasks 1.spring task supported by spring

Spring task is a scheduled task that comes with the above version of spring3. When realizing the function of scheduled task, you need to introduce spring-context package. At present, it supports two ways: xml and annotation.

1. Project actual combat

Because the xml approach is too old, let's take the annotations in the springboot project as an example.

The first step is to introduce spring-context-related dependencies into the pom.xml file.

Org.springframework spring-context

The second step is to annotate the springboot startup class with @ EnableScheduling.

@ EnableScheduling @ SpringBootApplication public class Application {public static void main (String [] args) {new SpringApplicationBuilder (Application.class) .web (WebApplicationType.SERVLET) .run (args);}}

The third step is to define timing rules using the @ Scheduled annotation.

Service public class SpringTaskTest {@ Scheduled (cron = "${sue.spring.task.cron}") public void fun () {System.out.println ("doSomething");}}

Step 4, configure the parameters in the applicationContext.properties file:

Sue.spring.task.cron=*/10 *?

This allows the fun method to be executed every 10 seconds.

2. Cron rules

In versions above spring4, the cron expression contains six parameters:

[seconds] [minutes] [hours] [date] [month] [week]

Several commonly used special symbols are also supported:

*: indicates that a task is triggered at any time

Indicates that the task is triggered at the specified time

-: indicates that the task is triggered within a period of time

/: indicates when the task starts and how often the task is triggered.

Represents two subexpressions for the day of the month and the day of the week, indicating that no value is specified.

The specific meaning of cron expression parameters:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Seconds. Value range: 0-59. *, -, -, / are supported.

Points. Value range: 0-59. Support *, -, -, /.

Value range: 0-23. *, -, -, / are supported.

Date. Value range: 1-31. *, -, -, / are supported. More than seconds, indicating that if the specified week is triggered, the configured date becomes invalid.

Month. Value range: 1-12. *, -, -, / are supported.

Week, value range: 1 for Sunday, 6 for Saturday, and so on. Support *, -, /,? More than seconds, indicating that if the specified date is triggered, the configured week becomes invalid.

Examples of common cron expressions:

0 001 *? Execute at zero o'clock on the 1st of each month

0 02 *? Every day at 2: 00 a. M.

0 02 *? Every day at 2: 00 a. M.

0 0Up 5 11 * *? 11:00-11:55 every day, every 5 minutes

018? * WED executes every Wednesday at 6: 00 p.m.

Spring task first parses and collects parameters in Scheduled annotations, including: cron expressions, through the processScheduled method of the ScheduledAnnotationBeanPostProcessor class.

Then in the afterPropertiesSet method of the ScheduledTaskRegistrar class, a single-threaded ThreadPoolExecutor execution task is initialized by default.

If you are interested in spring task, you can add me Wechat to talk to me privately.

Advantages and disadvantages of using spring task:

Advantages: spring framework comes with timing function, springboot is well encapsulated, it is very easy to open and define timing tasks, and supports complex cron expressions, which can meet most stand-alone business scenarios. When a single task is completed, the next task scheduling will be executed after the current scheduling is completed.

Disadvantages: default single-threading, if the previous task execution time is too long, it will affect the execution of later tasks. Cluster deployment is not supported, and data storage scheduling tasks cannot be done.

2.spring quart

Zquartz is an open source project of OpenSymphony open source organization in the field of Job scheduling. It is an open source task schedule management system developed by java.

What can quartz do?

Job scheduling: call job scripts of various frameworks, such as shell,hive, etc.

Scheduled tasks: perform the tasks you want to perform at a predetermined time.

The architecture diagram is as follows:

The main interfaces of quartz are as follows:

Scheduler stands for scheduling container, and multiple JobDetail and Trigger can be registered in one scheduling container.

Job stands for work, that is, the specific content to be carried out.

JobDetail represents a specific executable scheduler, and Job is what this executable scheduler is going to execute.

JobBuilder is used to define or build JobDetail instances.

Trigger stands for scheduling trigger and decides when to call it.

TriggerBuilder is used to define or build triggers.

JobStore is used to store the status of jobs and tasks during scheduling.

1. Project actual combat

Let's take springboot integration with quartz as an example.

The first step is to introduce quartz-related dependencies into the pom.xml file.

Org.springframework.boot spring-boot-starter-quartz

The second step is to create a real scheduled task execution class that inherits from QuartzJobBean.

Public class QuartzTestJob extends QuartzJobBean {@ Override protected void executeInternal (JobExecutionContext context) throws JobExecutionException {String userName = (String) context.getJobDetail () .getJobDataMap () .get ("userName"); System.out.println ("userName:" + userName);}}

The third step is to create the scheduler JobDetail and scheduler Trigger.

@ Configuration public class QuartzConfig {@ Value ("${sue.spring.quartz.cron}") private String testCron / * create a scheduled task * / @ Bean public JobDetail quartzTestDetail () {JobDetail jobDetail = JobBuilder.newJob (QuartzTestJob.class) .withIdentity ("quartzTestDetail", "QUARTZ_TEST") .usingJobData ("userName", "susan") .storeDuried () .build (); return jobDetail } / * create trigger * / @ Bean public Trigger quartzTestJobTrigger () {/ / execute CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule (testCron) every 5 seconds / / create trigger Trigger trigger = TriggerBuilder.newTrigger () .forJob (quartzTestDetail ()) .withIdentity ("quartzTestJobTrigger", "QUARTZ_TEST_JOB_TRIGGER") .withschedule (cronScheduleBuilder) .build (); return trigger;}}

Step 4, configure the parameters in the applicationContext.properties file:

Sue.spring.quartz.cron=*/5 *?

This allows you to execute the executeInternal method of the QuartzTestJob class every 5 seconds.

CronTrigger configuration format:

[seconds] [minutes] [hours] [day] [month] [week] [year]

The cron expression rules of spring quartz and spring task are basically the same, except that the later year is removed from the version above spring4, while the year of quartz's CronTrigger is optional, so I won't talk too much about it here.

Advantages and disadvantages of using spring quartz:

Advantages: the default is multi-thread asynchronous execution. For a single task, when the previous scheduling is not completed, the next scheduling time is due, another thread will start a new scheduling, and multiple tasks will not affect each other. Support for complex cron expressions that can be instantiated by clusters and support distributed deployment.

Disadvantages: the cost of implementing timing tasks is higher than that of spring task, which requires manual configuration of QuartzJobBean, JobDetail, Trigger and so on. Need to introduce a third-party quartz package, there is a certain learning cost. Parallel scheduling, failure handling strategy and dynamic slicing strategy are not supported.

four。 Distributed timing task 1.xxl-job

Xxl-job is a distributed task scheduling platform developed by Dianping (Xu Xueli). Its core design goal is rapid development, easy learning, lightweight and easy to expand. Now open source code and access to a number of companies online product lines, out of the box.

The xxl-job framework extends quartz, using a mysql database to store data, and built-in jetty to be invoked as a RPC service.

The main features are as follows:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

There are interface maintenance scheduled tasks and trigger rules, which are very easy to manage.

Ability to start or stop tasks dynamically

Support flexible expansion and reduction

Support task failure alarm

Support dynamic slicing

Support for failover

Rolling real-time log

Support for user and rights management

Management interface:

The overall architecture is as follows:

The diagram of the architecture using quartz is as follows:

Project actual combat

Xxl-admin management background deployment and mysql script execution and other preparatory work, I will not introduce too much, friends in need can talk to me privately, these are more inclined to the operation and maintenance of things.

Assuming that the previous work is already OK, then we need to:

The first step is to introduce xxl-job-related dependencies into the pom.xml file.

Com.xuxueli xxl-job-core

The second step is to configure parameters in the applicationContext.properties file:

Xxl.job.admin.address: http://localhost:8088/xxl-job-admin/ xxl.job.executor.appname: xxl-job-executor-sample xxl.job.executor.port: 8888 xxl.job.executor.logpath: / data/applogs/xxl-job/

The third step is to create a HelloJobHandler class that inherits the IJobHandler class:

JobHandler (value = "helloJobHandler") @ Component public class HelloJobHandler extends IJobHandler {@ Override public ReturnT execute (String param) {System.out.println ("XXL-JOB, Hello World."); return SUCCESS;}}

In this way, the scheduled task is configured.

It is recommended that scheduled tasks be deployed separately to another service, separate from the api service. According to my past experience, job will do batch operations on data in most cases. If the amount of data is too large, it may affect the memory and cpu resources of the service to a certain extent.

Advantages and disadvantages of using xxl-job:

Advantages: it has interface management timing task, supports flexible capacity expansion and reduction, dynamic slicing, fault transfer, failure alarm and other functions. Its function is very powerful, many large factories are in use, and can meet the vast majority of business scenarios.

Cons: like quartz, it uses database distributed locks to control that tasks cannot be repeated. In the case of a large number of tasks, there are some performance problems.

2.elastic-job

Elastic-job is a flexible distributed task scheduling system developed by Dangdang. It has rich and powerful functions. It uses zookeeper to achieve distributed coordination and achieve high availability and fragmentation of tasks. It is specifically developed for high concurrency and complex business scenarios.

Elastic-job is currently a sub-project of apache's shardingsphere project at http://shardingsphere.apache.org/elasticjob/.

Elastic-job after 2.x, there are two product lines: Elastic-Job-Lite and Elastic-Job-Cloud, and we generally use Elastic-Job-Lite to meet the demand. Elastic-Job-Lite is positioned as a lightweight decentralized solution, providing coordination services for distributed tasks in the form of jar packages, and externally relying only on Zookeeper.

The main features are as follows:

Distributed scheduling coordination

Elastic expansion and reduction

Failure transfer

Missed execution of job retrigger

Job fragmentation consistency to ensure that there is only one execution instance of the same shard in a distributed environment

Self-diagnose and fix problems caused by distributed instability

Support parallel scheduling

Overall architecture diagram:

Project actual combat

The first step is to introduce elastic-job-related dependencies into the pom.xml file.

Com.dangdang elastic-job-lite-core com.dangdang elastic-job-lite-spring

The second step is to add the ZKConfig class and configure zookeeper:

@ Configuration @ ConditionalOn_Expression ("'${zk.serverList} '.length () > 0") public class ZKConfig {@ Bean public ZookeeperRegistryCenter registry (@ Value ("${zk.serverList}") String serverList, @ Value ("${zk.namespace}") String namespace) {return new ZookeeperRegistryCenter (new ZookeeperConfiguration (serverList, namespace);}}

Third, define a class that implements the SimpleJob interface:

Public class TestJob implements SimpleJob {@ Override public void execute (ShardingContext shardingContext) {System.out.println ("ShardingTotalCount:" + shardingContext.getShardingTotalCount ()); System.out.println ("ShardingItem:" + shardingContext.getShardingItem ());}}

Step 4: add JobConfig configuration tasks:

@ Configuration public class JobConfig {@ Value ("${sue.spring.elatisc.cron}") private String testCron; @ Value ("${sue.spring.elatisc.itemParameters}") private String shardingItemParameters; @ Value ("${sue.spring.elatisc.jobParameters}") private String jobParameters =; @ Value ("${sue.spring.elatisc.shardingTotalCount}") private int shardingTotalCount; @ Autowired private ZookeeperRegistryCenter registryCenter @ Bean public SimpleJob testJob () {return new TestJob ();} @ Bean public JobScheduler simpleJobScheduler (final SimpleJob simpleJob) {return new SpringJobScheduler (simpleJob, registryCenter, getConfiguration (simpleJob.getClass (), cron, shardingTotalCount, shardingItemParameters, jobParameters));} private geConfiguration getConfiguration (Class

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.