Shulou(Shulou.com)06/02 Report--

When it comes to microservices architecture, the two inevitable topics are service governance and distributed transactions. The vertical separation of database and business module brings us the improvement of system performance, stability and development efficiency, but also introduces some more complex problems. For example, on the issue of data consistency, we can no longer rely on local transactions of database. For a series of cross-database write operations, how to ensure their atomicity is a problem that we have to face under microservice architecture.

1 Distributed Transaction Solutions

According to the characteristics of distributed system, different distributed transaction solutions are produced based on different consistency requirements, such as two-phase commit for strong consistency, flexible transaction and transaction message for final consistency, etc. There is no absolute good or bad of all kinds of schemes. We can't evaluate them without specific scenes, let alone make reasonable choices. When selecting distributed transaction solutions, we need to fully understand the principles and design intentions of various solutions, and then combine the actual business scenarios to make scientific and reasonable choices.

2 Strongly consistent solutions

2.1 two-phase commit

There are two roles in the two-phase commit algorithm: transaction coordinator and transaction participant. A transaction generally involves multiple transaction participants. The specific two-phase process is shown in the following figure:

The first stage: after the library writing operation is completed, the coordinator sends a Prepare message to all participants, asking whether the local transactions of each participant can be submitted, and the participants return yes or no to the coordinator according to their own situations;

Phase 2: After the coordinator receives feedback from all participants, if all returns are OK to commit, it sends a commit transaction command to all participants. As long as one participant returns a non-commit, a rollback command is sent to all participants. As shown below:

Figure 1 Two-phase submission

In the two-phase model described above, there may be cases where the coordinator or individual participants are down during the transaction submission process, but in most cases the nodes participating in the transaction can learn the transaction status by querying other nodes and make correct actions. But in extreme cases, transactions can be unknown. Let's analyze the following scenario: when the coordinator sends a commit instruction and the only participant who receives the commit instruction also goes down after completing the commit, no node knows whether the transaction should be committed or rolled back, and the transaction is in an unknown state, so in this extreme case, it may cause data inconsistency. In order to overcome the defect of two-phase, a three-phase commit protocol is proposed.

2.2 three-phase commit

Three-phase commit is to split the second phase into two phases: pre-commit and confirmation commit. In this way, no matter which node is down during the transaction commit process, as long as there is a surviving node in the pre-commit or commit state, we can determine that the transaction is committable (the first stage has confirmed that the transaction is committable), and if there is no node in either state, we will roll back the transaction.

Figure 2 Three-phase submission

From the above analysis, we can see that the final "commit" of both two-phase and three-phase is an extremely time-consuming operation, and even in distributed systems, the probability of failure is very small, so we can think that two-phase commit can basically guarantee the atomicity of distributed transactions.

3 landing plan

The XA specification is a distributed transaction specification based on the theoretical model of two-phase commit. The resource manager in the specification is equivalent to the transaction participant; the transaction manager is equivalent to the transaction coordinator. At present, many mainstream relational databases have implemented the XA interface.

When it comes to practical applications, we will find some problems with two-phase submission:

1. Database products need to ensure data completion, and writing needs locking, so the locking time of database resources may be too long in the whole distributed transaction coordination process, which is not suitable for business scenarios with high concurrency and long sub-transaction life cycle;

2. XA specification requires transaction manager to record transaction execution status locally, so transaction manager as stateful service does not support transaction remote recovery;

XA can ensure data consistency to the greatest extent, but performance degradation is very serious in high concurrency scenarios, so it is not recommended to use XA if it is not "strong consistency" in data consistency requirements.

3.1 Final consistency solution

In most of our business scenarios, the ultimate consistency of data is pursued, and the industry has also proposed many flexible transaction solutions, which can ensure the consistency of data to a large extent. We can weigh the use according to the actual scenario. There are many specific solutions, and the design ideas can be summarized into the following three models:

3.1.1 TCC（Try-Confirm-Cancel）

TCC divides transactions into three phases: Try, Confirm, Cancel.

1. Try phase: try to execute the business and reserve resources;

2. Confirm phase: confirm the execution of business and use resources of Try phase;

3. Cancel phase: cancel the execution of the service and release the resources reserved in the Try phase;

We use a business scenario of transfer and remittance to illustrate the specific process of TCC. For example: Zhang San transfers 100 yuan to Li Si. A transfer business consists of two local transactions: 1. Zhang San's account is deducted by 100 yuan;2. Li Si's account is increased by 100 yuan.

The successful transaction processing flow is shown in Figure 3:

Figure 3 Try-Confirm Transaction Successful Processing Flow

The transaction failure processing flow is shown in Figure 4:

Figure 4. Try-Cancel Transaction Success Process

Try phase:

1. Check Zhang San's account, deduct 100 yuan from the account meeting the requirements, and record the deduction event (reserve resources);

2. Check the validity of Li Si's account;

Confirm：

If Try succeeds, Li Si's account will be increased by 100 yuan, and the transaction will be completed.

Cancel：

If Try fails, Zhang San's account will be increased by 100 yuan, the deduction event record will be deleted (reserved resources will be released), and the transaction will be cancelled.

From a performance perspective, the TCC process does not lock resources and has little impact on system concurrency performance, except for some additional auxiliary operations. Note that there are two technical difficulties to ensure data consistency in this model:

1. A role like transaction manager is required to ensure the integrity of the TCC process;

2. Confirm and Cancel methods need to be idempotent (because of inevitable retry operations);

TCC is very intrusive to business, very unfriendly to RD students, and the cost of business transformation is quite high.

3.1.2 SAGA model

SAGA model divides a distributed transaction into several local transactions, each local transaction has its own execution module and compensation module. When any local transaction goes wrong, the previous transaction can be recovered by calling the corresponding compensation method, so as to achieve the final consistency of data. SAGA's transaction manager is responsible for executing compensation logic when a transaction fails, either by invoking inverse operations of the execution module (such as generating inverse SQL while executing subtransactions) or by invoking compensation methods provided by the business developer (which need to guarantee idempotent compensation).

It can be seen that although SAGA causes some intrusion to the business, when the TCC is much better, and the transaction manager can theoretically compensate backwards (undo all completed operations and restore to the transaction start state) or forward (continue to complete outstanding transactions, so that the business request is successfully processed, more in line with business expectations).

3.1.3 MQ Transaction Messages

MQ transaction messages simplify the distributed transaction model. The emphasis is no longer on ensuring the atomicity of all subtransactions, but on ensuring the atomicity of local transactions and sending MQ messages. We can use this feature to transform distributed transactions into local transactions and several operations to send MQ messages, and then ask consumers to ensure the success of consumption. MQ transaction messaging simplifies the distributed transaction model by removing the role of transaction manager from TCC and SAGA schemes, and is also the least intrusive and most friendly scheme (no compensation interface).

Of course, there are two basic premises:

1. MQ system ensures that messages can not be lost;

2. Consumers ensure that consumption is idempotent (it is difficult to avoid repeated consumption if it is guaranteed not to be lost).

It should be noted that MQ transaction messages simplify the transaction model and reduce business intrusion, so the guarantee of data consistency is relatively low.

4. summary

Although SAGA and TCC seem to guarantee the final consistency of data in flexible transaction solutions, the production environment of distributed systems is complex and changeable, and some cases can lead to the failure of flexible transaction mechanism. Therefore, no matter which solution is used, it needs the final bottom strategy, manual verification and repair of data.

Let's compare several distributed transaction solutions:

Consistency Assurance: XA > TCC = SAGA > Transaction Messages

Business-friendliness: XA > Transaction Messages> SAGA > TCC

Performance Loss: XA > TCC > SAGA = Transaction Message

Finally, when designing the system, we must combine the consistency requirements of the business itself and choose the appropriate solution. It can be seen that the higher the data consistency guarantee, the greater the development cost, maintenance difficulty and system performance loss. We must not blindly pursue the high-end scheme and over-design the system.

More free technical information and videos

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.