Shulou(Shulou.com)06/02 Report--

Background

Distributed transaction is not only a technical difficulty in enterprise integration, but also something that will be involved in every distributed system architecture, especially in micro-service architecture, it is almost inevitable.

ACID

Refers to the four basic elements of the correct execution of a database transaction:

Atomicity (Atomicity)

Consistency (Consistency)

Isolation (Isolation)

Persistence (Durability)

CAP

CAP principle, also known as CAP theorem, refers to the consistency (Consistency), availability (Availability), partition tolerance (Partition tolerance) in a distributed system. The CAP principle means that these three elements can only achieve two points at the same time, and it is impossible to take all three into consideration.

Consistency: whether all data backups in a distributed system have the same value at the same time.

Availability: whether the cluster as a whole can respond to read and write requests from clients after a failure of some nodes in the cluster.

Partition tolerance: in practical terms, partitioning is equivalent to the time limit for communication. If the system cannot achieve data consistency within the time limit, it means that a partition situation has occurred and a choice must be made between C and A for the current operation.

BASE theory

BASE theory is the result of a tradeoff between consistency and availability in CAP. The core idea of the theory is: we can not achieve strong consistency, but each application can adopt appropriate ways to achieve the final consistency of the system according to its own business characteristics.

Basically Available (basic available)

Soft state (soft state)

Eventually consistent (final consistency)

Solution

01

Two-phase commit (2PC)

Two-phase commit 2PC is one of the most powerful transaction types in distributed transactions. Two-stage commit is committed in two phases. The first phase asks whether each transaction data source is ready, and the second stage really commits the data to the transaction data source.

To ensure that the transaction satisfies the ACID, a Cooradinator is introduced. The other nodes are called Participant. The coordinator is responsible for scheduling the behavior of the participants and finally deciding whether they want to commit the transaction. The processing flow is as follows:

Stage one

A) the coordinator sends the transaction content to all participants, asks if the transaction can be submitted, and waits for a reply.

B) each participant performs transaction operations, recording undo and redo information in the transaction log (but not committing the transaction).

C) if the participant executes successfully, feedback yes to the coordinator, otherwise feedback no.

Stage two

If the coordinator receives a failed message from a participant or times out, a rollback message is sent directly to each participant; otherwise, a commit message is sent. The two situations are dealt with as follows:

Case 1: when all participants feedback yes, commit the transaction

A) the coordinator sends a request to all participants to formally submit the transaction (that is, the commit request).

B) the participant executes the commit request and releases the resources consumed throughout the transaction.

C) each participant feedback the completed message of ack (reply) to the coordinator.

D) upon receipt of ack messages from all participants, the coordinator completes the transaction commit.

Case 2: when a participant gives feedback to no, roll back the transaction

A) the coordinator sends a rollback request (i.e. rollback request) to all participants.

B) the participant uses the undo information in phase 1 to perform the rollback operation and release the resources consumed throughout the transaction.

C) each participant feedback the message completed by ack to the coordinator.

D) upon receipt of ack messages from all participants, the coordinator completes the transaction.

problem

1) performance problems: all participants are in a synchronous blocking state during the transaction commit phase, which takes up system resources, which can easily lead to performance bottlenecks.

2) Reliability issues: if the coordinator has a single point of failure, or if there is a failure, the provider will remain locked.

3) data consistency issues: in phase 2, if both the coordinator and the participants are dead, it may lead to data inconsistency.

Advantages: try to ensure the strong consistency of data, which is suitable for key areas with high requirements for strong consistency of data. (in fact, there is no 100% guarantee of strong consistency.)

Disadvantages: complex implementation, sacrifice availability, have a great impact on performance, and are not suitable for high-concurrency and high-performance scenarios.

02

Three-phase commit (3PC)

The three-phase commit is an improved version of the two-phase commit, and the most important thing for 3PC to solve is the problem that the coordinator and participants hang up at the same time, so 3PC divides the preparation phase of 2PC into two again, so three-phase commit. The processing flow is as follows:

Stage one

A) the coordinator sends a canCommit request with transaction content to all participants, asking if the transaction can be committed, and waits for all participants to reply.

B) after receiving the canCommit request, if the participant thinks that the transaction operation can be performed, the participant will report back to yes and enter the ready state, otherwise feedback no.

Stage two

According to the response of the participants, the coordinator has the following two possibilities.

Case 1: all participants feedback yes, and the coordinator pre-executes the transaction.

A) the coordinator sends a preCommit request to all participants and enters the preparatory phase.

B) after receiving the preCommit request, the participant performs a transaction operation and records the undo and redo information in the transaction log (but does not commit the transaction).

C) each participant feedback the ack response or no response to the coordinator and wait for the final instruction.

Case 2: as long as one participant gives feedback to the no, or the coordinator cannot receive feedback from all providers after waiting for the timeout, the transaction is interrupted.

A) the coordinator sends an abort request to all participants.

B) whether an abort request is received from the coordinator or a timeout occurs while waiting for the coordinator request, the participant interrupts the transaction.

Stage three

The real transaction commit at this stage can also be divided into the following two situations.

Case 1: all participants feedback the ack response to perform a real transaction commit

A) if the coordinator is working, a do Commit request is issued to all participants.

B) upon receipt of the do Commit request, the participant will formally perform the transaction commit and release the resources occupied during the entire transaction.

C) each participant feedback the message completed by ack to the coordinator.

D) upon receipt of ack messages from all participants, the coordinator completes the transaction commit.

Case 2: as long as one participant gives feedback to the no, or the coordination group cannot receive feedback from all providers after waiting for the timeout, the transaction is rolled back.

A) if the coordinator is working, send a rollback request to all participants.

B) the participant uses the undo information in phase 1 to perform the rollback operation and release the resources consumed throughout the transaction.

C) each participant feedback the message completed by ack to the coordination group.

D) after receiving the ack messages from all participants, the coordination group will complete the transaction rollback.

Advantages: compared with two-phase commit, three-phase commit reduces the blocking scope, and the coordinator or participant interrupts the transaction after waiting for a timeout. The single point problem of the coordinator is avoided. When there is a problem with the coordinator in phase 3, the participant continues to commit the transaction.

Disadvantages: the problem of data inconsistency still exists. When the participant waits for the do commite instruction after receiving the preCommit request, if the coordinator requests to interrupt the transaction, and the coordinator is unable to communicate with the participant normally, it will cause the participant to continue to submit the transaction, resulting in data inconsistency.

03

Compensation transaction (TCC)

TCC is a service-oriented two-phase programming model that adopts a compensation mechanism:

Conditions:

Need to implement confirmation and compensation logic

Idempotents need to be supported

Processing flow:

A) the Try phase is mainly about business system testing and resource reservation.

This stage is mainly completed:

Complete all business checks (consistency).

Reserve necessary business resources (quasi-isolation).

Try attempts to execute the business.

B) the Confirm phase is mainly about confirming and submitting the business system.

When the Try phase executes successfully and the Confirm phase begins, the default Confirm phase does not go wrong. That is, as long as Try succeeds, Confirm will succeed.

C) in the Cancel phase, the business executed in the event of a business execution error and needs to be rolled back is cancelled and reserved resources are released.

Advantages:

Performance improvement: the granularity of the specific business to control the resource lock becomes smaller, and the whole resource will not be locked.

Final consistency of data: based on the idempotency of Confirm and Cancel, it ensures that the transaction is finally confirmed or cancelled to ensure the consistency of the data.

Reliability: the problem of single point failure of the coordinator of the XA protocol is solved, the whole business activity is initiated and controlled by the main business party, and the business activity manager becomes multi-point and is introduced into the cluster.

Disadvantages: the Try, Confirm and Cancel operation functions of TCC should be realized according to the specific business, which has a high degree of business coupling and increases the development cost.

04

Local message table (message queue)

Its core idea is to split the distributed transaction into local transactions for processing.

In the scheme, by creating an additional transaction message table in the consumer, the consumer processes the business and records the transaction message in the local transaction, polls the data of the transaction message table to send the transaction message, and the provider consumes the transaction in the transaction message table based on the message middleware.

Conditions:

The service consumer needs to create a message table to record the message status.

Service consumers and providers need to support idempotency.

Compensation logic is required.

Timed threads start on each node to check for incomplete or failed messages and reissue messages, that is, retry mechanism and idempotent mechanism.

Processing flow:

1. The service consumer submits the business data with the message and initiates the transaction.

two。 The message is sent to the service provider via MQ, and the service consumer waits for the processing result.

3. The service provider receives the message, completes the business logic and notifies the consumer of the message that has been processed.

Fault-tolerant processing is as follows:

When something goes wrong in step 1, the transaction rolls back, which means nothing happens.

When there is an error in the processing of steps 2 and 3, because the message is saved in the consumer table, it can be re-sent to MQ for retry.

If there is a processing error in step 3 and it is a business failure, the service provider sends a message to notify the consumer that the transaction failed, and at this point the consumer initiates the rollback transaction to roll back the logic.

Advantages: the reliability of message data is realized from the point of view of application design and development, and the reliability of message data does not depend on message middleware, which weakens the dependence on MQ middleware features.

Disadvantages: binding to specific business scenarios, strong coupling, not public. Message data is in the same library as business data and occupies business system resources. When the business system uses the relational database, the performance of the message service will be limited by the concurrent performance of the relational database.

MQ transaction messages (final consistency)

MQ, which supports transactional messages, supports transactional messages in a manner similar to two-phase commit.

The distributed transaction scheme based on MQ is actually the encapsulation of the local message table, which is based on the internal MQ, and other protocols are basically the same as the local message table.

Conditions:

A) compensation logic is required

B) Business processing logic requires idempotency

Processing flow:

C) consumers send half messages to MQ.

D) after MQ Server persists the message, it confirms to the sender ack that the message was sent successfully.

E) consumers begin to execute transaction logic.

F) the consumer submits a second confirmation or rollback to the MQ Server based on the execution result of the local transaction.

G) when MQ Server receives the commit status, it marks the half message as deliverable.

H) the service provider receives the message and executes local business logic. Returns the processing result.

Advantages:

The message data is stored independently to reduce the coupling between the business system and the message system.

The throughput is better than the local message table scheme.

Disadvantages:

One message delivery requires two network requests (half message + commit/rollback).

You need to implement the message lookback interface.

05

Sagas transaction model (ultimate consistency)

Saga mode is a distributed asynchronous transaction, an ultimate consistent transaction, and a flexible transaction. There are two different ways to implement saga transactions. The two most popular ways are:

Event / choreography Choreography: in the absence of a central coordinator (no single point of risk), each service generates and listens to events from other services and decides whether action should be taken.

The first service of the implementation performs a transaction and then publishes an event. This event is monitored by one or more services that perform local transactions and publish (or do not publish) new events. When the last service executes a local transaction and does not publish any events, it means the end of the distributed transaction, or that the event it publishes is not heard by any Saga participant means the transaction is over.

Processing flow:

The order service saves the new order, sets the state to the pengding pending state, and publishes an event named ORDER_CREATED_EVENT.

The payment service listens to the ORDER_CREATED_EVENT and publishes the event BILLED_ORDER_EVENT.

The inventory service monitors BILLED_ORDER_EVENT, updates inventory, and releases ORDER_PREPARED_EVENT.

The freight service monitors the ORDER_PREPARED_EVENT and then delivers the product. Finally, it releases ORDER_DELIVERED_EVENT.

Finally, the order service listens for ORDER_DELIVERED_EVENT and sets the status of the order to concluded completion.

Suppose the inventory service fails during the transaction. Roll back:

Inventory service generates PRODUCT_OUT_OF_STOCK_EVENT

The ordering service and payment service supervisor heard about the above incident of inventory service:

① payment service will be refunded to the customer.

The ② order service sets the order status to failure.

Pros: event / choreography is a natural way to implement the Saga pattern; it is simple, easy to understand, does not require much effort to build, and all participants are loosely coupled because they are not directly coupled to each other. If your transaction involves 2 to 4 steps, it may be very appropriate.

Command / Coordination orchestrator: the central coordinator is responsible for centralizing the decision-making and business logic sequencing of events.

The saga coordinator orchestrator communicates with each service in a command / reply manner, telling them what to do.

The order service saves the pending status and requires the order Saga coordinator (OSO) to start the order transaction.

OSO sends an execution collection order to the collection service, and the collection service replies to the Payment Executed message.

OSO sends a prepare order command to the inventory service, and the inventory service will reply to the OrderPrepared message.

OSO sends an order shipping order to the shipping service, and the shipping service will reply to the Order Delivered message.

The OSO order Saga coordinator must know in advance the process required to perform the create order transaction (obtained by reading the BPM business process XML configuration). If there is any failure, it is also responsible for coordinating the distributed rollback by sending commands to each participant to undo the previous operation. When you have a central coordinator to coordinate everything, rollback is much easier, because the coordinator defaults to executing the forward process, but only the reverse process is needed for the rollback.

Advantages:

Avoid circular dependencies between services because the saga coordinator invokes the saga participant, but the participant does not invoke the coordinator.

Centralized choreography of distributed transactions.

You only need to execute the command / reply (in fact, the reply message is also an event message) to reduce the complexity of the participants.

When adding new steps, the transaction complexity remains linear and the rollback is easier to manage.

If you want to change the target of a second transaction before the first transaction has been executed, you can easily pause it on the coordinator until the first transaction is completed.

Cons: too much logical risk is concentrated in the coordinator.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.