Shulou(Shulou.com)06/02 Report--

How to carry out distributed transaction Seata Saga mode and three modes, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

I. the background of distributed transactions

1.1 Evolution of distributed architecture-horizontal splitting of databases

Ant Financial Services Group's business database is a single database and single table at first, but with the rapid development of the scale of business data, the amount of data is getting larger and larger, and the single database and single table has gradually become a bottleneck. So we split the database horizontally and split the original single database and single table into database slices.

As shown in the following figure, after dividing the database and tables, the write operations that can be completed on one database may span multiple databases, which leads to the problem of cross-database transactions.

Cdn.nlark.com/yuque/0/2019/png/226702/1561951736610-8e3271ef-c433-4bab-9973-68dab108ee30.png ">

1.2 Evolution of distributed Architecture-split of Business Services

In the early stage of business development, the single business system architecture of "a big piece of cake" can meet the basic business needs. However, with the rapid development of business, the visit volume and business complexity of the system are growing rapidly. single system architecture has gradually become the bottleneck of business development, and the demand to solve the problem of high coupling and scalability of business system is becoming stronger and stronger.

As shown in the following figure, Ant Financial Services Group splits a single business system into multiple business systems in accordance with the design principles of Service-oriented Architecture (SOA), which reduces the coupling between each system and makes different business systems focus on their own business, which is more conducive to business development and system capacity expansion.

After the business system is split according to services, a complete business often needs to invoke multiple services, so how to ensure the data consistency among multiple services has become a difficult problem.

II. Theoretical basis of distributed transactions 2.1 two-phase commit protocol

Two-phase commit protocol: the transaction manager coordinates the resource manager in two phases. The first stage prepares resources, that is, reserves the resources needed by the transaction. If each resource manager reserves resources successfully, the second phase resource commit is carried out. Otherwise, the coordination resource manager rolls back the resources.

2.2 TCC

TCC (Try-Confirm-Cancel) is actually a service-oriented two-phase commit protocol. Business developers need to implement these three service interfaces. In the first phase, the service is programmed by the business code to call the Try interface for resource reservation. The Try interfaces of all participants are successful, and the transaction manager commits the transaction and calls each participant's Confirm interface to actually commit the business operation, otherwise call each participant's Cancel interface to roll back the transaction.

2.3 Saga

Saga is a compensation protocol. In Saga mode, there are multiple participants in a distributed transaction, and each participant is a correction compensation service, which requires users to implement their forward operation and reverse rollback operation according to the business scenario.

In the process of distributed transaction execution, the forward operations of each participant are executed in turn, and if all forward operations are executed successfully, then the distributed transaction commits. If any of the forward operations fail, the distributed transaction goes back to perform the reverse rollback of the previous participants, rolling back the committed participants and bringing the distributed transaction back to its original state.

Saga theory comes from Sagas, a paper published by Hector & Kenneth 1987.

Saga forward services and compensation services also need to be implemented by business developers.

3. Detailed explanation of Seata and its three modes 3.1 introduction to distributed transaction Seata

Seata (Simple Extensible Autonomous Transaction Architecture, simple Extensible Autonomous transaction Framework) is a distributed transaction solution jointly opened by Ant Financial Services Group and Alibaba in January 2019. Seata open source for about half a year, there have been more than 11000 star, the community is very active. We warmly welcome all of you to participate in Seata community building and build Seata into an open source distributed transaction benchmark product.

Seata: https://gitee.com/seata/seata

3.2 distributed transaction Seata product module

As shown in the following figure, there are three major modules in Seata, namely TM, RM, and TC. TM and RM are integrated with the business system as the client of Seata, and TC is deployed independently as the server of Seata.

In Seata, the execution flow of a distributed transaction:

TM starts distributed transactions (TM registers global transaction records with TC)

Arrange intra-transaction resources such as databases and services according to business scenarios (RM reports resource readiness status to TC)

TM ends distributed transaction, transaction phase ends (TM informs TC to commit / roll back distributed transaction)

TC aggregates transaction information and determines whether distributed transactions are committed or rolled back

TC notifies all RM commit / rollback resources that the transaction phase two ends

3.3 distributed transaction Seata solution

Seata will have four distributed transaction solutions, namely AT mode, TCC mode, Saga mode and XA mode.

3.3.1 AT mode

In January this year, Seata opened up the AT model. AT mode is a non-intrusive distributed transaction solution. In AT mode, users only need to focus on their own "business SQL", and the user's "business SQL" as a phase, the Seata framework will automatically generate two-phase commit and rollback operations of the transaction.

How the AT model is non-intrusive to the business:

One stage:

At one stage, Seata will intercept the "business SQL", first parse the SQL semantics, find the business data that the "business SQL" wants to update, save it as "before image" before the business data is updated, then execute the "business SQL" to update the business data, save it as "after image" after the business data is updated, and finally generate a row lock. The above operations are all done in a database transaction, which ensures the atomicity of the one-phase operation.

Two-phase submission:

If the second phase is a submission, because the "business SQL" has been submitted to the database in the first phase, the Seata framework only needs to delete the snapshot data and row locks saved in the first phase and complete the data cleaning.

Two-phase rollback:

If the second phase is a rollback, Seata needs to roll back the "business SQL" that has been executed in the first phase to restore the business data. The rollback method is to restore business data with "before image". However, dirty writing should be verified before restore. Comparing "database current business data" and "after image", if the two data are exactly the same, it means that there is no dirty writing, and business data can be restored. If there is inconsistency, dirty writing needs to be transferred to manual processing.

The one-phase and two-phase commit and rollback of AT mode are automatically generated by Seata framework. Users only need to write "business SQL" to easily access distributed transactions. AT mode is a distributed transaction solution without any intrusion into the business.

3.3.2 TCC mode

In March 2019, Seata opened up the TCC model, which was contributed by Ant Financial Services Group. TCC mode requires users to implement Try, Confirm and Cancel operations according to their own business scenarios; the transaction initiator executes the Try mode in the first phase, the Confirm method in the second phase commit, and the Cancel method in the second phase rollback.

The three methods of TCC are described:

Try: detection and reservation of resources

Confirm: the business operation performed is submitted; Try is required to succeed Confirm must be successful

Cancel: reserved resource release

Ant Financial Services Group's practical experience in TCC

1 TCC design-the business model is designed in two phases:

When users connect to TCC, the most important thing is to consider how to split their business model into two stages.

Take the "money deduction" scenario as an example. Before connecting to TCC, the deduction of account A can be completed by a SQL that updates the account balance. However, after connecting to TCC, users need to consider how to split the deduction operation that can be completed in one step into two stages and implement it into three methods, and ensure that the first-stage Try is successful and the second-stage Confirm will be successful.

As shown in the figure above, the Try method, as an one-phase preparation method, requires resource checking and reservation. In the case of withholding money, what Try needs to do is to check whether the account balance is sufficient and reserve transfer funds by freezing the transfer funds of An account. After the execution of the Try method, the balance of account An is still 100, but 30 yuan of it has been frozen and cannot be used by other transactions.

The two-stage Confirm method performs the real deduction operation. Confirm will use the funds frozen in the Try phase to perform account deductions. After the implementation of the Confirm method, the 30 yuan frozen in account An in one stage has been deducted, and the balance of account A has become 70 yuan.

If the second stage is a rollback, you need to release the 30 CNY frozen in the first stage Try in the Cancel method to make account A return to its original state, and all 100 CNY will be available.

When users access TCC mode, the most important thing is to consider how to split the business model into two stages and implement three methods of TCC, and to ensure the success of Try Confirm must be successful. Compared with AT mode, TCC mode is intrusive to business code, but TCC mode does not have the global row lock of AT mode, and TCC performance will be much higher than AT mode.

2 TCC Design-allow empty rollback:

The Cancel interface needs to be designed to allow empty rollback. When the Try interface is not received due to packet loss, the transaction manager will trigger a rollback, which will trigger the Cancel API. When Cancel executes, it needs to return a successful rollback when it finds that there is no corresponding transaction xid or primary key. Let the transaction service manager think that it has been rolled back, otherwise it will keep retrying, and Cancel has no corresponding business data to roll back.

3 TCC design-anti-suspension control:

Suspension means that Cancel executes before Try interface, because Try times out due to network congestion, transaction manager generates rollback, triggers Cancel interface, and finally receives Try API call, but Cancel arrives before Try. According to the previous logic that allows empty rollback, the rollback will return success, and the transaction manager thinks that the transaction has been rolled back successfully, then the Try interface should not be executed at this time, otherwise data inconsistency will occur. Therefore, we record the transaction xid or business primary key before the Cancel empty rollback is successful, indicating that the record has been rolled back. The Try API first checks the transaction xid or business primary key if it has been marked as successful. The business operation of the Try is not performed.

4 TCC design-idempotent control:

Idempotency means that for the same system, using the same conditions, the impact of one request and repeated requests on system resources is the same. Because the network jitter or congestion may time out, the transaction manager will retry the resources, so it is very likely that a business operation will be called repeatedly. In order not to occupy resources for many times because of repeated calls, idempotent control is needed when the service is designed. Usually, we can use transaction xid or business primary key to determine the retry.

3.3.3 Saga mode

Saga mode is a long-term transaction solution that Seata will open source soon, which will be mainly contributed by Ant Financial Services Group. In Saga mode, there are multiple participants in a distributed transaction, and each participant is a correction compensation service, which requires users to implement their forward operations and reverse rollback operations according to the business scenario.

In the process of distributed transaction execution, the forward operations of each participant are executed in turn, and if all forward operations are executed successfully, then the distributed transaction commits. If any of the forward operations fail, the distributed transaction goes back to perform the reverse rollback of the previous participants, rolling back the committed participants and bringing the distributed transaction back to its original state.

Distributed transactions in Saga mode are usually event-driven and executed asynchronously among participants. Saga mode is a long transaction solution.

1 Saga mode usage scenario

Saga mode is suitable for business systems that have long business processes and need to ensure the final consistency of transactions. Saga mode will commit local transactions in the first stage, and can ensure performance in the case of unlocked and long processes.

Transaction participants may be services of other companies or legacy systems that cannot be modified and provide the interfaces required by TCC, but can use the Saga pattern.

The advantages of the Saga model are:

One-stage commit of local database transactions, no locks, high performance

Participants can use transaction-driven asynchronous execution with high throughput

Compensation service is the "reverse" of forward service, which is easy to understand and easy to implement.

Disadvantages: Saga mode cannot guarantee isolation because the local database transaction has been committed in one phase and there is no "reservation" action. The response to the lack of isolation will be discussed later.

2 Saga implementation based on state machine engine

At present, there are generally two kinds of Saga implementations, one is implemented through event-driven architecture, and the other is based on annotations and interceptors to intercept business forward service implementation. At present, Seata is implemented by event-driven mechanism. Seata implements a state machine, which can arrange the call flow of the service and the compensation service of the forward service, generate a state diagram defined by the json file, and the state machine engine drives the diagram to run. When an exception occurs, the state machine triggers rollback and executes the compensation service one by one. Of course, the user can customize the decision under what circumstances to trigger the rollback. The state machine can realize the requirements of service orchestration, and it supports the functions of single item selection, concurrency, asynchronism, sub-state machine invocation, parameter transformation, parameter mapping, service execution state judgment, exception capture and so on.

Principle of 3 state machine engine

The basic principle of the state machine engine is that it is based on the event-driven architecture, each step is executed asynchronously, and the flow between steps through the event queue greatly improves the system throughput. A transaction log is recorded when each step is executed, which is used for rollback in the event of an exception, and the transaction log is recorded in the same database as the business table to improve performance.

4 State Machine engine Design

The state machine engine is divided into three-tier architecture design, the bottom layer is the "event-driven" layer, which implements the thread pool of EventBus and consumer events, and is a Pub-Sub architecture. The second layer is the "process controller" layer, which implements a minimalist process engine framework, which drives an "empty" process execution. "empty" means that it does not care about what the process node does, it only executes the process method of each node, and then executes the route method to flow to the next node. This is a general framework, based on these two layers, developers can implement any process engine. The top layer is the "state machine engine" layer, which implements the "behavior" and "routing" logic code of each state node, provides API and state diagram repository, as well as some other components, such as expression language, logic calculator, pipeline generator, interceptor, configuration management, transaction logging and so on.

5 experience in Saga service design

Similar to TCC, Saga's forward and reverse services also need to follow the following design principles:

1) Saga service design-allow null compensation

2) Saga service design-anti-suspension control

3) Saga service design-idempotent control

4) Saga Design-Custom transaction recovery Strategy

As mentioned earlier, the Saga mode does not guarantee transaction isolation, and dirty writing may occur in extreme cases. For example, in the case that the distributed transaction is not committed, the data of the previous service is modified, and the exception of the latter service needs to be rolled back, which may be because the data of the previous service cannot be compensated after being modified. One way to deal with this could be to "retry" to continue to complete the distributed transaction. Because the entire business process is choreographed by a state machine, you can continue to retry even after the event. Therefore, users can configure whether the transaction strategy of the process is priority "rollback" or "retry" according to the business characteristics. When the transaction times out, the Server side will continue to retry according to this strategy.

Since Saga does not guarantee isolation, we need to achieve the principle of "prefer long money to short money" when designing our business. long money refers to the situation where there is more money from our point of view when something goes wrong, and less money is short money, because if the long money can be refunded to the customer, while the short money may not be recovered, that is to say, in the business design, the customer's account must be deducted before entering the account. If there is an overlay update due to isolation, there will be no shortage of money.

6 Saga implementation based on annotations and interceptors

Another implementation of Saga is based on annotation + interceptor. Seata is not currently implemented. You can understand it by looking at the pseudocode above. The annotation @ SagaCompensable is defined on the one method, and the compensation method used to define the one method is the compensateOne method. Then define the @ SagaTransactional annotation on the business process code processA method, start the Saga distributed transaction, intercept each forward method to trigger a rollback operation when an exception occurs, and call the compensation method of the forward method.

7 comparison of the advantages and disadvantages of two kinds of Saga implementation

The two Saga implementations have their own disadvantages. The following table is a comparison:

The biggest advantage of the state machine engine is that it can improve system throughput through asynchronous execution of event-driven methods, and can achieve service orchestration requirements. In the case of lack of isolation in Saga mode, there can be one more "forward retry" thing recovery strategy. The biggest advantage of annotation plus interceptor is that it is easy to develop and low learning cost.

Seata is positioned as a distributed full-scenario solution, and there will be distributed transaction implementation in XA mode in the future. Each model has its own applicable scenario. AT mode is a non-invasive distributed transaction solution, which is suitable for scenarios where you do not want to transform the business, with almost zero learning cost. TCC mode is a high-performance distributed transaction solution, which is suitable for scenarios with high performance requirements, such as core systems. Saga mode is a long transaction solution, which is suitable for business systems that have long business processes and need to ensure the final consistency of transactions. Saga mode will commit local transactions in the first stage, without locks, and can ensure performance in the case of long processes. It is often used in channel layer and integration layer business systems. Transaction participants may be services of other companies or legacy systems that cannot be modified and provide the interfaces required by TCC, or Saga patterns can be used.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.