Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge of "analyzing distributed timed task elastic-job". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

In the actual development, we often encounter scenarios such as running batches on a regular basis and performing reconciliation operations on a daily basis. Timing tasks such as spring timing framework, ScheduledExecutorService and so on are usually used. However, these are only suitable for a single machine, and the problem of repeated execution of timing tasks will occur in the case of multiple sections, so it is necessary to use distributed timing tasks to solve the problem. Distributed timing tasks not only solve the above problems, but also provide the advantages of slicing processing to improve processing efficiency, distributed scheduling coordination, flexible capacity expansion and reduction, failure transfer and so on.

What are the common distributed timing tasks?

At present, there are many open source distributed timing tasks. Several commonly used ones are listed below:

Elastic-job: a flexible distributed task scheduling system developed by Dangdang, with rich and powerful functions, uses zookeeper to achieve distributed coordination, achieves high availability and fragmentation of tasks, and can support cloud development.

Xxl-job: a distributed task scheduling platform released by Xu Xueli, an employee of Dianping, in 2015, it is a lightweight distributed task scheduling framework. Its core design goal is rapid development, easy learning, lightweight and easy to expand.

Saturn: is a distributed scheduled task scheduling platform independently developed by VIPSHOP. It is developed based on Dangdang's elastic-job version 1 and can be well deployed to docker containers.

Among them, elastic-job (E-job) and xxl-job (X-job) are widely used. There is no more specific discussion on the technology selection of the two. The following provides a comprehensive comparison of the two for reference, which can be combined with the actual business and technical scenarios for selection.

2. What are the slicing strategies for elastic-job assignments?

Combined with the actual scene, choosing the appropriate job slicing strategy and slicing processing can improve the execution efficiency of the task. Here are the sharding strategies provided by elastic-job

AverageAllocationJobShardingStrategy fragmentation strategy

OdevitySortByNameJobShardingStrategy fragmentation strategy

RotateServerByNameJobShardingStrategy fragmentation strategy

Custom sharding policy

1. AverageAllocationJobShardingStrategy sharding strategy

Com.dangdang.ddframe.job.lite.api.strategy.impl.AverageAllocationJobShardingStrategy

Policy description:

The slicing strategy based on the average allocation algorithm is also the default slicing strategy. When the number of jobs is divisible by the number of servers,

If the shard is not divisible, the extra shards that cannot be divisible will be appended to the server with a small sequence number (the ip address comes first).

Disadvantages: in the average sharding strategy, when the number of shards is less than the number of job servers, jobs will always be assigned to the server at the front of the ip address, resulting in the server at the bottom of the IP address idle.

2. OdevitySortByNameJobShardingStrategy fragmentation strategy

Com.dangdang.ddframe.job.lite.api.strategy.impl.OdevitySortByNameJobShardingStrategy

Policy description:

The core idea of this strategy is to use IP ascending / descending algorithm to realize slicing according to the odd and even hash value of job name. if the hash value of job name is odd, IP ascending order, and if the hash value of job name is even, IP descending order. In this way, it is used to balance the load of different job fragments to different servers.

If the hash value of the job name is even, the IP descending algorithm is used to achieve slicing, which avoids the problem that the server at the bottom of the IP address is idle when using the average allocation algorithm.

The average algorithm is used internally.

3. RotateServerByNameJobShardingStrategy fragmentation strategy

Com.dangdang.ddframe.job.lite.api.strategy.impl.RotateServerByNameJobShardingStrategy

Policy description:

The slicing strategy that rotates the server list according to the hash value of the job name is also calculated by average slicing.

4. Custom sharding policy

You can customize your sharding policy according to your needs by implementing the JobShardingStrategy API and the sharding method. The following is how to switch the custom allocation policy in Spring.

This is the end of the content of "analyzing distributed timed tasks elastic-job". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.