Shulou(Shulou.com)06/02 Report--

What this article shares with you is about the four solutions of distributed transactions for microservices. The editor thinks it is very practical, so I share it with you. I hope you can learn something after reading this article. Let's take a look at it with the editor.

Distributed transaction

Distributed transaction refers to the participants of the transaction, the server that supports the transaction, and the resource server is located on different nodes of the distributed system, usually a distributed

Operations on multiple data sources or business systems are involved in things.

Typical distributed transaction scenario: cross-bank transfer involves invoking two remote bank services

CAP theory

CAP theory: a distributed system can not meet the three basic requirements of consistency, availability and partition fault tolerance at the same time, and can only meet two of them at most.

Item

Consistency (C): the characteristic of whether data can be consistent across multiple copies.

Availability (A): means that the services provided by the system must be consistently available, and the request for each user always returns the result within a limited time.

Sometimes it is considered that the system is not available.

Partition fault tolerance (P): when a distributed system encounters any network partition failure, it still needs to be able to provide services that meet consistency and availability.

Unless the entire network environment fails.

Application of CAP Theorem

Abandon P (CA): if you want to avoid the problem of partition fault tolerance in the system, a relatively simple way is to put all the data (or related to things first)

All of the data are placed on a distributed node, so that although there is no guarantee that 100% of the system will not go wrong, at least it will not encounter the negative effects caused by the network partition.

Ring

Abandon A (CP): the practice is that once the system encounters a network partition or other failure, the affected service needs to wait for a certain amount of time to apply the waiting period of the system

Unable to provide normal services, that is, unavailable

Abandon C (AP): abandoning consistency here does not mean that there is no need for data consistency at all. It means to give up the strong consistency of data and retain the ultimate consistency of data.

BASE theory

BASE is basically available, soft state, and ultimately consistent. It is the result of consistency and availability permissions in CAP, and is based on the CAP theorem.

It is thought that even if strong consistency cannot be achieved, each application can adopt appropriate ways to achieve final consistency according to its own business specificity.

2PC submission

The two-phase commit protocol divides the commit process of a transaction into two stages: committing the transaction request and executing the transaction commit.

Phase 1: commit transaction request

Transaction inquiry: the coordinator sends the transaction content to all participants, asks if the transaction commit operation can be performed, and starts waiting for each participant to respond.

Execute transaction: each participant node performs transaction operations and records Undo and Redo information in the transaction log

If the participant successfully executes the transaction operation, it feedback to the coordinator Yes response, indicating that the thing can be executed, and if the transaction is not executed successfully, feedback to the coordinator.

No response, indicating that the transaction cannot be executed

The stage in which some of the two stages are committed overnight is called the voting phase, in which each participant's vote indicates whether the next transaction commit can continue.

Phase 2: perform transaction commit

If the coordinator's feedback from all participants is a Yes response, then the transaction commit is performed.

Send submit request: the coordinator sends a Commit request to all participant nodes

Transaction commit: after the participant receives the Commit request, the transaction commit operation is formally performed, and after the commit is completed, the transaction occupied during transaction execution is discarded.

Transaction resources

Feedback transaction submission result: after completing the transaction submission, the participant sends an ACK message to the coordinator

Complete the transaction: after receiving the ACK message from all participants, the coordinator completes the transaction

Interrupt a transaction

If any participant feedback the No response to the coordinator, or if the coordinator cannot receive the feedback from all participants after waiting for the supermarket, then

Interrupt the transaction.

Send a rollback request: the coordinator sends a Rollback request to all participant nodes

Transaction rollback: after receiving the Rollback request, the participant will perform a transaction rollback operation using the Undo information recorded during the phase, and after completing the rollback

Releases the resources occupied during the execution of the transaction after.

Feedback on the transaction rollback result: participants send an ACK message to the coordinator after the transaction rollback is completed

Interrupt transaction: after receiving the ACk message from all participants, the coordinator completes the transaction interruption,

Advantages and disadvantages

The principle is simple and the realization is convenient.

The disadvantages are synchronous blocking, single point problem, brain fissure, conservatism.

3PC submission

Three-phase commit, also known as three-phase commit protocol, is an improved version of two-phase commit (2PC).

Unlike two-phase commit, three-phase commit has two change points. Introduce the timeout mechanism. At the same time, the timeout mechanism is introduced in both the coordinator and the participants. In the first place

Insert a preparation phase between the phase and the second phase. It ensures that the status of the participating nodes is consistent before the final submission phase.

There are three stages of submission: CanCommit, PreCommit and DoCommit.

Seata distributed transaction scheme

Seata is an open source distributed transaction solution dedicated to providing high-performance and easy-to-use distributed transaction services. Seata will provide users with

AT, TCC, SAGA and XA transaction modes to create an one-stop distributed solution for users.

Seata terminology

TC: transaction coordinator. Maintain the state of global and branch transactions and drive global transaction commit or rollback.

TM: transaction manager. Define the scope of a global transaction: start a global transaction, commit, or roll back a global transaction

RM: manage the resources for branch transactions, talk to TC to register branch transactions and report the status of branch transactions, and drive branch transactions to commit or rollback.

Seata's 2PC scheme

Phase 1: business data and rollback log records are committed in the same local transaction, releasing local locks and connection resources.

The second phase: the submission is asynchronized and completed very quickly. The rollback is reverse compensated through an one-stage rollback log.

Before committing a first-phase local transaction, you need to make sure that you get the global lock first. Cannot get global lock, cannot commit local transaction.

The attempt to get the global lock is limited to a certain range, which is abandoned, and the local transaction is rolled back, releasing the local lock.

On the basis that the database local transaction isolation level reads committed or above, the default global isolation level for Seata (AT mode) is read uncommitted

If the application is in a specific scenario, it must require that the global read has been submitted. The current way of Seata is through the proxy of the SELECT FOR UPDATE statement.

Analysis of Seata execution process

Each RM uses DataSourceProxy to link data paths to use ConnectionProxy, and the purpose of using data sources and data agents is in the first phase

Put undo_log and business data in a local transaction commit, thus saving that as long as there is a business operation, there must be a undo_log

The modified values before and after data modification are stored in the first stage undo_log to make a good distinction for the transaction rollback, so the branch transaction has been mentioned as soon as the first stage is completed.

If it is handed in, the lock resource will be released.

TM starts the global transaction, puts the XID global transaction ID in the transaction context, and passes the XID to the downstream branch transaction through the feign call, each branch transaction

Associate your own Branch ID branch transaction ID with XID

In the second stage, the global transaction commits, and TC informs each branch participant to commit the branch transaction. In the first phase, the branch transaction has already been committed. Here, all the participants need to do is

You can delete the undo_log and execute it asynchronously. The second phase can be completed soon.

If a branch transaction is abnormal, the second phase rolls back the global transaction, and TC notifies each branch participant to roll back the branch transaction through XID and

Branch-ID finds the corresponding rollback log, rolls back the reverse SQL generated by the log and executes it to complete the branch transaction rollback before

The actual combat case list of Seata

Github.com/seata/seata...

Github.com/seata/seata...

TCC distributed transaction

TCC is a service-oriented two-stage programming model, and its Try, Confirm and Cancel,3 methods are all implemented by business coding.

TCC requires that each branch transaction implement three operations: preprocess the Try, confirm the Confirm, and undo the Cancel.

Try operation for business inspection and resource reservation

Confirm does business confirmation operation.

Cancel implements a rollback operation that is the opposite of Try.

TM first initiates all branch transaction Try operations. If the Try operation of any branch transaction fails, TM will initiate Cancel operations of all branch transactions.

If all Try operations are successful, TM will initiate Confirm operations for all branch transactions, and if the Confirm/Cancel operation fails, TM will retry.

The three stages of TCC

The Try phase is to do business check (consistency) and resource reservation (isolation). This phase is only a preliminary operation, and it can be integrated with the subsequent Confirmy.

Business logic of

The Confirm phase is to confirm the commit, and the Try phase starts to execute Confirm after the successful execution of all branch transactions. In general, TCC is considered as

There is no error in the Confirm phase, that is, as long as the Try is successful, the Confirm will be successful. If there is a real error in the Confirm phase, you need to introduce a retry mechanism or labor.

Deal with

The Cancel phase is to cancel the branch transaction and release the reserved resources when the business execution error needs to be rolled back to the state. In general, using TCC

The Cancel phase must also be true. If there is a real error in the Cance phase, a retry mechanism or manual processing should be introduced.

TM transaction manager: TM transaction manager can be implemented as an independent service, or global transaction initiator can play the role of TM. TM is independent for public purposes.

Using components is to consider the system structure and software reuse.

TM generates a global transaction record when initiating a global transaction. The global transaction ID runs through the entire distributed transaction call chain and is used to record transaction context, track and record.

State, failure for Confirm and cacel requires a retry, so idempotent is required

Three kinds of exception handling of TCC

Idempotent treatment

Because of network jitter and other reasons, the distributed transaction framework may repeatedly call the two-phase interface of a branch transaction in the same distributed transaction. So branch transactions

The two-phase interface Confirm/Cancel needs to be able to guarantee idempotency. If the two-stage interface cannot guarantee idempotency, serious problems will arise, resulting in resources.

Re-use or re-release, resulting in business failure.

For idempotent problems, the usual method is to introduce idempotent fields to prevent replay attacks. For the idempotent problem in the distributed transaction framework, we can also present

This sharp weapon.

The timing of insertion of idempotent records is the participant's Try method, where the branch transaction state is initialized to INIT. Then when the second phase of the Confirm/Cancel executes

Its status is set to CONFIRMED/ROLLBACKED when the.

When TC calls the two-phase API repeatedly, participants will first obtain the corresponding records of the transaction status control table to view their transaction status. If the status is already

CONFIRMED/ROLLBACKED, which means that the participant has done his or her part, does not need to execute it again, and can directly return the idempotent successful result.

To TC to help it advance distributed transactions.

Empty rollback

When the participant Try method is not called, the two-phase Cancel method is called, and the Cancel method needs to have a way to identify whether the Try is executed at this time. If the Try has not been executed, the Cancel operation is invalid, that is, the Cancel belongs to an empty rollback; if the Try has been executed, then the normal rollback logic is executed.

To deal with the problem of empty rollback, you need to give participants a way to identify whether the first-phase Try has been executed in the two-phase Cancel method. Obviously, yes.

Continue to use the transaction status control table to achieve this function.

When the Try method is executed successfully, a record is inserted indicating that the branch transaction is in the INIT state. So when the two-phase Cancel method is called later,

It can be judged by querying the corresponding records of the control table. If the record exists and the status is INIT, the first phase has been successfully executed, and the rollback operation can be performed normally.

Do, release the reserved resources; if the record does not exist, the first phase has not been executed, this time is an empty rollback, no resources are released.

Resource suspension

Problem: after the TC rollback transaction call completes the empty rollback in the second phase, the first phase executes successfully

Solution: the transaction state control record is used as the control means, the record is inserted when no record is found in the second stage, and the existence of the record is checked during the first stage execution.

Comparison between TCC and 2PC

2PC is usually at the DB level across libraries, while TCC is processed at the application level and needs to be implemented through business logic. The advantages of this distributed transaction are

You can let the application define the granularity of data operations, making it possible to reduce lock conflicts and improve throughput

The deficiency is that it is very intrusive to the application, and each branch of the business logic needs to implement three operations of Try,confirm,cancel. Besides, in fact,

At present, it is also quite difficult, and different rollback strategies need to be implemented according to the network status and the different failure causes of the system failure.

Implementation of TCC case list in Hmily Framework

# account Atry: idempotency of try try suspension processing check whether the balance is sufficient for 30 CNY deduction of 30 CNY confirm: null processing is usually considered that confirm is error-free cancel: cancel idempotence cacel empty rollback processing increases the available balance by 30 CNY Rollback operation # account B try: empty processing can be done. Idempotency of confirm: confirm is officially increased by 30 yuan cancel: empty processing is available.

RocketMQ to achieve the final consistency of reliable messages

The ultimate consistency of reliable messages is to ensure the consistency of messages delivered from the producer to the consumer through the message middleware.

RocketMQ mainly solves two functions: the atomicity of local transactions and message sending. Reliability of messages received by transaction participants

Reliable message final consistency transaction is suitable for scenarios with long execution cycle and low real-time requirements. With the introduction of message mechanism, synchronous transaction operations become based on message execution.

To avoid the influence of synchronous blocking operations in distributed transactions, and realize the decoupling of the two services

Best effort notification

What's the difference between best effort notification and reliable message consistency?

Reliable message consistency, the initiating notifier needs to ensure that the message is sent and sent to the receiving notifier, and the reliability of the message is guaranteed by the originating notifier.

Best effort notification, the initiating notifier tries his best to notify the receiving party of the business processing result, but the message may not be received, and the notification needs to be received at this time.

The party actively calls the interface of the initiating party to query the service, and the key to the reliability of the notification lies in the receiving party.

Application scenarios of both

Reliable message consistency is concerned with the transaction consistency of the transaction process, completing the transaction asynchronously.

The best effort notification focuses on the notification transaction after the transaction, that is, the reliable notification of the transaction result.

Ack Mechanism based on MQ to realize Best effort Notification

Using the ack mechanism of MQ, the MQ sends a message notification to the recipient, and the initiator sends the ordinary message to MQ.

Receive notification, monitor MQ, receive message, and respond to ACK after business processing is completed.

If the recipient does not respond to the ACK, the MQ will repeat the notification and gradually enlarge the notification interval according to the way of the time interval.

This scheme is suitable for notification between internal microservices, but not for external platforms.

Option 2: add a notification service area for notification, which is applicable when external third parties are provided.

Comparative Analysis of distributed transaction schemes

One of 2PC's biggest complaints is a blocking protocol. RM waits for TM's decision after performing a branch transaction, and the service blocks locked resources. Because of its resistance

The congestion mechanism and the worst-case time complexity are high, so this design can not meet the need to scale with the increase of the number of services involved in the transaction, and it is difficult to be used with high concurrency.

And distributed services with longer life cycle of sub-transactions

If you compare the processing flow of TCC transactions with the two-phase commit of 2PC, 2PC is usually at the DB level across libraries, while TCC is processed at the application level, which requires communication.

Through the business logic to achieve. The advantage of this distributed transaction is that it allows the application to customize the granularity of data operations, reducing lock conflicts and improving throughput

Maybe. The deficiency is that it is very intrusive to the application, and each branch of the business logic needs to implement three operations. In addition, it is also difficult to realize.

It is necessary to implement different strategies according to different causes of failure, such as network status, system failure and so on.

Reliable message final consistency transaction is suitable for scenarios with long execution cycle and low real-time requirements. With the introduction of message mechanism, synchronous transaction operations become based on message execution.

The asynchronous operation avoids the influence of synchronous blocking operation in distributed transactions, and realizes the decoupling of two services, typical scenarios: registration points, login delivery

Coupons, etc.

Best effort notification is one of the lowest requirements in distributed transactions. it is suitable for some services with low time sensitivity of final consistency, and allows the initiator to notify the business to deal with the loss.

Failure, actively deal with the failure after receiving the notification, no matter how the initiating party handles the result, it will not affect the subsequent processing of the receiving party.

The notifier needs to provide an API to query the execution, which is used to receive the result of proofreading by the notifier. Typical application scenarios: bank notification, payment result notification, etc.

Best effort notification of 2PC TCC reliable information

The above is what the four solutions of micro-service distributed transactions are like. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.