Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you about how to deal with multiple data sources in the sub-database and table. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

After the transformation of the system by sharding, the original single database will evolve into multiple databases. How to ensure the atomicity and consistency of multiple data sources to operate at the same time is a problem that has to be considered. Generally speaking, there are three ways of transaction processing for a distributed system: distributed transaction, transaction based on Best Efforts 1PC mode and transaction compensation mechanism. Let's analyze these three processing methods one by one.

Distributed transaction

This is the most well-known multi-source transaction mechanism. This article does not intend to introduce distributed transactions too much. Readers can refer to this article: research on distributed transactions, two-phase commit, one-phase commit, Best Efforts 1PC mode, and transaction compensation mechanism. Here I just want to analyze the pros and cons of distributed transactions.

Advantages:

1. Based on two-phase commit, the atomicity of cross-database operation is guaranteed to the maximum, and it is the strictest way to implement transactions in distributed systems.

two。 The implementation is simple and the workload is small. Because most application servers and some independent distributed transaction coordinators do a lot of encapsulation work, the difficulty and workload of introducing distributed transactions into the project are basically negligible.

Disadvantages:

The mortal enemy of the system's "horizontal" scalability. A distributed transaction based on two-phase commit needs to coordinate among multiple nodes when committing a transaction, which limits the time of committing the transaction and objectively prolongs the execution time of the transaction. this will lead to an increase in the probability of conflicts and deadlocks when transactions access shared resources. with the increase of database nodes, this trend will become more and more serious. As a result, it becomes the shackle of the horizontal scalability of the system at the database level, which is the main reason why many Sharding systems do not adopt distributed transactions.

Transaction based on Best Efforts 1PC mode

Different from the two-phase commit adopted by distributed transactions, Best Efforts 1PC mode adopts one-phase commit, which sacrifices the security of transactions in some special cases (downtime, network outage, etc.), but achieves good performance, especially eliminating the shackles of horizontal scaling. Distributed transactions in Spring, with and without XA explains the Best Efforts 1PC pattern in detail, and the Demo code provided in this paper directly gives an example of multi-data source transaction management for one-phase commit in Spring environment. However, it should be noted that the original example is based on the previous version of spring 3.0. if you use spring 3.0, you will get the following error: java.lang.IllegalStateException: Cannot activate transaction synchronization-already active. If you use spring 3.0, you need to refer to the implementation of spring-data-neo4j. Given the performance advantages of the Best Efforts 1PC pattern and its relatively simple implementation, it is adopted by most sharding frameworks and projects.

Transaction compensation mechanism

For those systems that have high requirements for performance but not high requirements for consistency, they often do not require the real-time consistency of the system, as long as the final consistency can be achieved within an allowable time period. this makes the transaction compensation mechanism a feasible scheme. Transaction compensation mechanism was originally proposed in the processing of "long transactions", but it also has a good reference significance for distributed systems to ensure consistency. Generally speaking, unlike the way in which a transaction is rolled back immediately after an error occurs in execution, transaction compensation is a measure of checking and remediation afterwards, and it only expects a final and consistent result within an allowable time period. The implementation of transaction compensation is closely related to the system business, and there is no standard way to deal with it. Some common implementations are: reconciliation of data, comparison based on logs, periodic synchronization with standard data sources, and so on.

Distributed transaction, the strictest transaction implementation, but performance is a big problem; Best Efforts 1PC mode, the balance between performance and transaction reliability, support system horizontal scaling, in most cases is the most suitable choice; transaction compensation mechanism can only be applied to systems that do not require high transactionality and allow data to be "ultimately consistent", at the expense of real-time consistency to get the maximum performance return.

The above is the editor for you to share how to deal with the transactions of multiple data sources in the sub-database and sub-table. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.