Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to solve the transaction problem after the database is divided into databases and tables. Many people may not know much about it. In order to make you understand better, the editor summed up the following contents for you. I hope you can gain something according to this article.

I. Overview

With the development of time and business, the amount of data in the table in the database will become larger and larger, accordingly, the cost of data operation, addition, deletion, modification and query will become more and more large. Therefore, some of these large tables are split into multiple tables in multiple databases.

This article is based on the asynchronous guarantee of non-transactional messages to complete the transaction problems in the sub-library and sub-table.

Second, the problem needs to be solved

2.1 original transaction

Because after the sub-database and sub-table, the new table is in another database, how to ensure the transactionality of the main database and sub-database is a problem that must be solved.

Solution: map the logic of operating the database to a pipeline record by creating a pipeline table in the main library. When the whole big transaction is completed (the pipeline is inserted into the pipeline meter), then the pipeline is executed in other ways to ensure the final consistency.

2.2 running water

The so-called pipelining can be understood as a transaction message.

Above, by creating a pipelining table in the database and using a pipelining record to represent a business processing logic, a pipelining must eventually be executed correctly. Therefore, when extracting a piece of business code from the pipeline, you must consider:

Pipelined delay processing. Pipelining is not processed in real time, but is executed asynchronously with pipelined actuators. Therefore, if in the original logic, you need to pay special attention to whether the subsequent process is dependent on the pipeline in real time (for example, the pipelining result will be used to do some calculations in the subsequent business logic).

Disorderly processing of running water. Ensure that even if the later generated pipelining is executed first, there will be no problems.

The running water finally succeeds. For each inserted pipeline, the pipeline must be executed successfully.

Therefore, when extracting the pipeline:

The simpler the flow treatment, the better.

The less dependence on churn processing, the better.

The extracted pipeline has no real-time dependence in the business logic.

2.3 pipelined treatment completed

Because the pipeline meter is placed in the original database, and the operation sub-library is completed after the pipeline processing is completed, if the sub-library operation is completed to update the old table pipeline message, then it is a boast library transaction, how to ensure that the update and sub-library of the pipeline state are also in the same transaction?

The solution is to create a flow table in the sub-library, and when the loss processing is completed, instead of updating the status of the old table, insert it into the sub-library flow table,

The benefits of this:

Generally speaking, a unique index is made on the pipeline, so if the pipeline is executed repeatedly, when inserting the sub-library pipeline table, the whole transaction will be rolled back because the unique index test fails. (of course, you can also make an idempotent judgment before processing the pipeline.)

In this way, by judging whether the main library pipeline is in the sub-library, we can judge whether a stream has been executed or not.

III. Basic framework of pipelining processor

In fact, the pipeline processor does not contain any business-related processing logic, and the core functions are:

Inform the business access party when to handle what kind of running water

Verify the success of the pipeline execution

Note: the pipeline actuator does not know what logic the pipeline represents, but needs the business system to identify it and execute the corresponding business logic.

3.1 pipelined execution task

The scheduling task of pipeline processing is to scan the pipeline to be processed, and then tell the business system which pipeline to execute.

The schematic diagram is as follows:

3.2 pipelined verification task

The pipelining verification task is to compare the pipelining records in the main database and the sub-database, reprocess the unsuccessful pipelining notification business system, and issue an alarm if multiple retries fail.

Process diagram:

Fourth, why not use transaction messages

Since it is an existing project (Internet finance, so any message loss or message processing failure is absolutely not tolerated), there is one reason why transaction messages are not used.

Additional message queuing is required to increase the complexity of the system, as well as additional logical guarantees and handling when message queuing communication fails.

In fact, 1 is not the main reason, but because the transaction message requires manual commit and rollback (not needed to use the database), then the problem is that the transaction in spring is transitive, and when our transaction message is committed is a big problem. For example, A.A () is originally a transaction. But A.A () is called in another transaction B.b (). Is the transaction message commit in A.A () or B.B ()?

After reading the above, do you have any further understanding of how to solve the transaction problem after the database sub-table? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.