Shulou(Shulou.com)06/03 Report--

This article mainly explains "what are the advantages of using event-driven instead of scheduled tasks". The content of the explanation in this article is simple and clear, and it is easy to learn and understand. let's study and learn what are the advantages of using event-driven instead of scheduled tasks.

I. the use of scheduled tasks

In the current system design, the timing task exists as a very important component. Let me give you two scenarios as examples throughout this article.

1. Summary of document details #

For example, the sales details of an e-commerce supermarket need to classify and summarize the total sales according to the previous day's data according to certain conditions (location, channel, even suppliers, goods, etc.).

For this scenario, using scheduled tasks is almost a mindset: choose a time when the system is relatively idle in the early morning, and pull all the data that need to be summarized through scheduled task scheduling for classification and summary.

two。 Automatic change of contract status #

For example, the supermarket and the supplier sign a rebate contract, and the effective time field and current status field above the contract. The default state is "to be effective", and once the current time reaches the effective time, the contract status changes to "effective".

This kind of scenario also has the obvious characteristic of one-size-fits-all according to time, so using scheduled tasks is almost the only choice.

II. Event-driven use

What's wrong with using scheduled tasks? Why should I recommend the event mechanism here?

Next I will redesign the functionality in the previous example with event-triggered logic that is more in line with our way of thinking. Through the comparison of different ways, you should be able to feel the direct and obvious difference between them.

1. Summary of document details #

As mentioned earlier, summarization means getting the data that needs to be processed, grouping the data according to specific summary conditions and then compressing them.

There are several key pieces of information: data to be processed by ①, and ② to compress data in groups according to conditions.

So let's start with the first point of view: which data needs to be processed. You don't have to think about it, you don't have to summarize the data the day before. In fact, what I want to ask is not this, but how to get the data. You can't handle it until you get it.

Quite simply, with scheduled tasks, we can filter out the data by time period and whether or not it has been processed. But how to have a large amount of data, such as tens of millions or even billions of dollars a day, it may be difficult to pull it out at once and sort it in memory.

Some people will say: no, we have experience in dealing with huge amounts of data: slicing processing. If we use sharding processing, we can get all the stores that are open through a basic data interface, such as 100000, and then divide them into several groups according to certain rules. each group is processed in different program instances (or sequentially in the same program). If it is a relational database, adding an index to the store can basically solve the problem.

Then there is another question to consider: the daily sales volume of stores is not fixed, and even some stores have no sales volume for a day, while the strategy for slicing stores is fixed, and which stores are fixed every day. The problem caused by this is that the amount of data in different fragments may be very different, and the calculation pressure of the fragments is also different.

If we do not use scheduled tasks, we have the following options to choose from, and I think it is better than scheduled tasks.

a. Delay summary #

Whenever a store writes data to the system, the system needs to determine whether the current data is written for the first time of the day. If this is the first time to write, create a delay summary event (the specific implementation can be timed mq or timed thread pool, but it is best not to use the database, otherwise we need a scheduled task to scan the database). In this way, stores without business volume will not be summarized.

In order to reduce the pressure on the system, our specific delay time can also be separated so that all the stores do not have to gather together.

However, using this scheme, compensation must be made accordingly. For example, using Java thread pooling or akka acotr, if the system restart information is lost; if mq is used, message consumption may also fail. So we need monitoring and alarm mechanisms to assist.

b. Real-time summary #

Whenever the data is written, determine the low summary dimension of the data, and then write the summary record. If the summary record already exists, update it directly.

The main consideration in this process is concurrency security. We can use distributed locks or database optimistic locks, or even mysql's on duplicate key update (see [https://www.manxi.info/mysqlDuplicate]). Other problems, such as peak elimination, can be judged in combination with the scene.

two。 Change of contract status #

The sign of using scheduled tasks in this scenario is that all records that hit a rule at a certain point in time are processed. So we can create a scheduled task in the early hours of each morning to determine if any contracts need to be valid today (or the flag expires). But in fact, there may only be one day in a year when this task really works, because other dates completely miss the record-it's a waste of resources!

For this scenario, we can learn the lazy cleanup of redis's cleaning strategy for expired keys. We do not need a mechanism to change the status of the contract at a precise (precise) time, but if the contract does not need to be hit, then its state has always been the same in the database: even after the expiration date, our connection to the database is still not valid. It is only when the business logic hits the contract that the contract needs to be found in the returned data for modification.

This method can effectively reduce the rotation empty in the system logic, but when querying the records, it is necessary to find the records of the former state. Because the contract that we need to hit is already in force, using the lazy strategy, we need to find the contract to be effective together, and then remove the expired contract from the contract that is already in force, and find the contract that has become effective from the contract that has become effective; at this time, we need to change the status of both expired and effective contracts. Of course, we can use asynchronous logic to handle state changes, because even if the process fails, the result of the next query will not be affected.

Event-driven benefits

From the above example, we can feel that there are actually advantages and disadvantages of using event triggering mechanism instead of scheduled tasks. The positive point is that the event mechanism is on demand, come as soon as you need it, and don't come if you don't need it. The drawback is that a set of auxiliary logic is generally needed to maintain the normal operation of the event mechanism: but what kind of logic does not need a compensation mechanism? It's just that some of them are mature and ready-made, while others need us to redevelop.

In essence, scheduled task is a stateless behavior, which is not in line with the idea of object-oriented. Event triggering simulates the process of people dealing with a thing.

For example, how do the operation and maintenance personnel deal with the summary of records before there is a management letter? If you wait until the next day to deal with the list squeezed the day before, this is the timing task. If the number of orders is too large to be processed manually at once, multiple people are needed to deal with it together, which is a distributed task. If more than one person can't handle it well for a while, and it takes a long time (the manual pressure is coming up), they may adapt to deal with one order, otherwise they will be idle for most of the day, so the pressure will come down. As for the system, real-time processing can effectively reduce the sudden rise of system pressure.

In fact, the manual processing of the backlog of documents is still not the prototype of the timing task, only because the manual can handle it, it is the expression of human nature rather than the transformation of the model. It is difficult to find a prototype of timing tasks in humans. Although humans now use alarm clocks, the task of alarm clocks is to remind and repeat tasks.

Simulate the handling of the contract again.

Assuming that there is no regulatory letter, the contract searcher estimates that there will be several squares, one waiting to take effect, one that is in effect, and one that has expired. When he needs to find a contract, he goes to the first two squares and puts the effective ones from the first to the second and the expired ones from the second to the third.

Is it possible for him to move these contracts ahead of time? It's possible, but it's totally unnecessary: it's still human-he's too lazy. And after he moved it today, there may be no need to use the contract all day today, so why doesn't he move tomorrow? So there is not a suitable time to move, just use the time to move.

Thank you for your reading. the above is the content of "what are the advantages of using event-driven instead of scheduled tasks". After the study of this article, I believe you have a deeper understanding of the advantages of using event-driven instead of scheduled tasks, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.