Shulou(Shulou.com)06/03 Report--

This article focuses on "how to use distributed transaction 2PC, 3PC model", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to use the distributed transaction 2PC and 3PC model.

What is a transaction?

A transaction is the smallest unit of work of a database operation, a set of operations that can no longer be divided, and a series of operations performed as a single logical unit of work. These operations are submitted to the system as a whole, either all or not

A transaction has four characteristics, namely, Atomicity, Consistency, Isolation and Durability, which is referred to as the ACID feature of a transaction.

How to guarantee the ACID feature of the transaction?

Atomicity: intra-transaction SQL either succeeds or fails at the same time, based on undo log (undo log) implementation

Consistency: the transfer of a system from one correct state to another, guaranteed by the application through AID, can be said to be the core feature of the transaction.

Isolation (Isolation): controls the visibility of data when transactions are executed concurrently, based on locks and multi-version concurrency control (mvcc)

Persistence (Durability): storage will not be lost after submission, based on redo log (redo log) implementation

This article mainly introduces distributed transaction 2PC and 3PC, and describes the contents of redo, undo log, mvcc and lock in detail later.

In the early days, our application was still a single project, so we operated on a single database, in which case we called it a local transaction. The ACID of local transactions is generally supported at the database level, such as the MySQL database that is commonly used in our work.

Usually, when we operate the MySQL client, MySQL implicitly commits transactions automatically, so daily work does not involve manual writing transaction creation, commit, rollback and other operations. If you want to experiment with lock, MVCC and other features, you can create multiple sessions and use begin, commit, rollback and other commands to experiment with the data between different transactions to see if the execution results are consistent with what you want.

We usually use Spring-encapsulated transactions when developing project code, so we don't manually write methods such as commit, rollback, and so on for database transactions (except in a few cases). Here we use native JDBC to write a sample code to help you understand how to guarantee the four features of ACID through transactions.

Connection conn =.; / / get the database connection conn.setAutoCommit (false); / / Open the transaction try {/ /. Execute sql conn.commit (); / / commit transaction} catch (Exception e) {conn.rollback (); / / transaction rollback} finally {conn.close (); / / close link}

Imagine whether it is painful to write repeated methods to create transactions, commit, rollback and so on every time you perform database operations. How does Spring automatically help us manage transactions? in Spring projects, we generally use two ways to manage transactions, programmatic transactions and declarative transactions.

Spring is used in the project to manage transactions, either by annotating @ Transactional on interface methods, or by using AOP to configure faceted transactions. In fact, the two methods are more or less the same, except that @ Transactional has a finer granularity and relies on AOP in principle, as an example

@ Service public class TransactionalService {@ Transactional public void save () {/ / Business Operation}}

TransactionalService is created by Spring to put a proxy object into the container. The created proxy object is equivalent to the following class

Public class TransactionalServiceProxy {private TransactionalService transactionalService; public TransactionalServiceProxy (TransactionalService transactionalService) {this.transactionalService = transactionalService;} public void save () {try {/ / start transaction operation transactionalService.save ();} catch (Exception e) {/ / rollback} / / commit transaction}} if an exception occurs

The sample code looks simple and clear, but the real code generation code comparison is much more complicated. With regard to the transaction manager, Spring provides the interface PlatformTransactionManager, which contains two important implementation classes

DataSourceTransactionManager: supports local transactions, and internally opens, commits and rolls back transactions through java.sql.Connection

JtaTransactionManager: used to support distributed transactions, which implements the JTA specification and uses the XA protocol for two-phase commit

Through these two implementation classes, we know that the programmatic transactions and declarative transactions we use usually rely on the implementation of local transaction management, and Spring also supports distributed transactions. There is a lot of information about the support of JTA distributed transactions on the Internet, so I won't repeat them here.

What is a distributed transaction

We use local transactions in daily business code all the time, and it's not difficult to understand. However, with the popularity of SOA and micro services, our single business system is usually split into multiple systems. in order to cater to the changes of the business system, the database is also split with the business.

For example, taking the school management system as an example, it may be divided into student services, curriculum services, teacher services, and so on, and the database may also be divided into multiple databases. When this happens, when different services are deployed to the server, it is possible to face the following service invocations

ServiceA services need to operate the database to execute local transactions, and at the same time need to call ServiceB and ServiceC services to initiate transaction calls. How to ensure that the transactions of the three services either succeed or fail together, and how to ensure the transaction ACID feature of the request initiated by the user? There is no doubt that this is a distributed transaction scenario, where a single local transaction of the three services cannot guarantee the entire requested transaction.

There are many solutions in distributed transaction scenarios. According to different categories, strong consistency solution and final consistency solution are subdivided into 2PC, 3PC, TCC, reliable messages.

Several solutions are widely used in the industry, such as Ali's RocketMQ transaction message, Seata XA pattern, and reliable message model. However, distributed transactions without exception operate multiple databases directly or indirectly, and the use of distributed transactions also brings new challenges, that is, performance problems. If the performance is degraded in order to ensure the strong consistency of the distributed transaction or the final consistency of the compensation scheme, the loss will undoubtedly outweigh the gain for the normal business.

DTP Model and XA Specification

The X/Open organization defines a distributed transaction model (DTP) and a distributed transaction protocol (XA). DTP consists of the following model elements

AP (Application application): used to define transaction boundaries (that is, to define the beginning and end of transactions) and to operate on resources within transaction boundaries

TM (Transaction Manager transaction Manager): responsible for assigning the unique identity of the transaction, monitoring the progress of transaction execution, and responsible for transaction commit, rollback, etc.

RM (Resource Manager Explorer): such as databases, file systems, etc., and provides ways to access resources

CRM (Communication Resource Manager Communication Resource Manager): controls communication between distributed applications within a TM domain (TM domain) or across TM domains

CP (Communication Protocol Communication Protocol): provides the underlying communication services between distributed application nodes provided by CRM

In the DTP distributed transaction model, the basic components need to cover AP, TM, RMS (no CRM, CP is also possible), as shown in the following figure

XA specification

The most important function of XA specification is to define the interface between RM (Resource Manager) and TM (transaction Manager). In addition, the XA specification not only defines the interactive interface between 2PC, but also optimizes 2PC.

Sort out the relationship among DTP, XA and 2PC

DTP defines the role model in distributed transactions, and specifies that the control of global transactions requires the use of 2PC protocol to ensure data consistency.

2PC is the abbreviation of Two-Phase Commit, that is, two-phase commit. It is an algorithm designed to ensure atomicity and consistency of all nodes in the distributed system architecture, especially in the field of database. At the same time, 2PC is also considered as a consistency protocol, which is used to ensure the consistency of data in distributed systems.

XA specification is a distributed transaction processing specification proposed by X/Open. XA specification defines the interface that needs to be used in 2PC (two-phase commit protocol), that is, the interaction between RM and TM in the figure above. 2PC and XA are the most easily confused. It can be understood that the DTP model defines the interface specification for communication between TM and RM called XA, and then the relational database (such as MySQL) based on the XA specification proposed by X/Open (the core depends on the 2PC algorithm) is called the XA scheme.

2PC consistency algorithm

When an application (AP) initiates a transaction operation that needs to span multiple distributed nodes, each distributed node (RM) knows whether the result of its transaction operation is successful or failed, but can not get the operation results of other distributed nodes. In order to ensure the ACID feature of transaction processing, it is necessary to introduce a component called "coordinator" (TM) to implement unified scheduling and distributed execution logic.

The coordinator is responsible for scheduling the behavior of the distributed nodes participating in the overall transaction and finally deciding whether these distributed nodes should commit or roll back the transaction. Therefore, based on this idea, two distributed consistency algorithm protocols, two-phase commit and three-phase commit, are derived. The second phase refers to the preparation phase and the submission phase. Let's first look at what has been done in the preparation phase.

2PC-preparation phase

The first phase of the two-phase commit is also called the "voting phase", in which each participant votes to indicate whether he or she will continue with the next transaction commit step.

Transaction inquiry: the coordinator sends transaction content to all participants participating in this distributed transaction, asks if the transaction commit operation can be performed, and then starts waiting for the response of each participant.

Execute transaction: the participant receives the transaction request from the coordinator, executes the corresponding transaction, and writes the contents to the Undo and Redo logs

Return response: if each participant executes the transaction, the coordinator Yes response is fed back; if each participant fails to successfully execute the transaction, the coordinator No response is returned

If all participants in the first phase return a successful response, then proceed to the transaction commit step, otherwise the distributed transaction returns with failure. Take the MySQL database as an example. In the first phase, the transaction manager (TM) issues a prepare (prepare commit) request to all the databases involved (RM). After receiving the request, the database performs data modification and logging processing, changes the transaction state to "committable" after processing, and finally returns the result to the transaction processor.

2PC-submission phase

The commit phase is divided into two processes: one is that each participant normally executes the transaction commit process and returns a Yes response, indicating that each participant voted successfully; the other is that each participant has failed to execute and returns a No response or timeout, which will trigger a global rollback, indicating the failure of distributed transaction execution.

Perform transaction commit

Interrupt a transaction

Perform transaction commit

Assuming that the feedback the coordinator receives from all participants is a Yes response, then the transaction commit operation will be performed

Transaction commit: the coordinator sends a Commit request to all participant nodes. After receiving the Commit request, each participant commits the local transaction and releases the transaction resources occupied during the transaction execution cycle after the commit is completed.

Complete transaction: after each participant completes the transaction submission, sends an Ack response to the coordinator, who receives the response and completes the distributed transaction.

Interrupt a transaction

If any transaction participant node feedback the No response to the coordinator (note that the No response here refers to the first phase), or after the wait timeout, the coordinator does not receive feedback from all participants, then the transaction interruption process will occur.

Transaction rollback: the coordinator issues a Rollback request to all participants. After receiving the rollback request, the participant uses the undo log written in the first phase to perform the rollback of the transaction and releases the occupied resources after the rollback transaction is completed.

Interrupt transaction: after completing the transaction rollback, the participant sends an Ack message to the coordinator. After the coordinator receives the Ack message from the transaction participant, the transaction interrupt is completed.

Advantages and disadvantages of 2PC

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

2PC commit divides the transaction process into two stages: voting and execution. The core idea is to try and commit each transaction. The advantage of 2PC is obvious, that is, it is simple in principle and easy to implement. Simplicity also means that many places are not perfect. Here are three core flaws.

Synchronous blocking: no matter in the first phase or in the second phase, all participant resources and coordinator resources are locked, and only when all nodes are ready, the transaction coordinator will notify the global commit. The resource is not released until the participant commits the local transaction. This process will be long and have a great impact on performance.

Single point of failure: if there is a problem with the coordinator, the entire two-phase commit process will not work. In addition, if the coordinator fails in the second phase, then other participants will be in a state of locking transaction resources

Data inconsistency: when the coordinator sends a Commit request to all participants in the second phase, a local network exception occurs or the coordinator itself crashes before sending the Commit request, resulting in only some participants receiving the Commit request, then the received participants will commit the transaction, resulting in data inconsistency

Because of the simplicity and convenience of 2PC, the above situations such as synchronous blocking, single point of failure and data inconsistency will occur, so an improvement has been made on the basis of 2PC and a three-phase commit (3PC) has been implemented.

There are many limitations in using 2PC. First of all, the database needs to support XA specification, and the performance and data consistency data are not friendly, so although XA schema is supported in Seata, AT schema is the main push.

3PC consistency algorithm

Three-phase commit (3PC) is an improved version of two-phase commit (2PC) and introduces two new features

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Both the coordinator and the participants introduce the timeout mechanism to solve the problem of synchronous blocking of 2PC and prevent transaction resources from being permanently locked.

The second phase is evolved into three phases, and the first "preparation phase" in the two-phase commit protocol is divided into two, forming a new transaction processing protocol composed of three phases: CanCommit, PreCommit and do Commit.

The detailed submission process of 3PC will not be discussed here. The greatest advantage of 3PC over 2PC is that it reduces the blocking range of participants and continues to reach agreement after the coordinator has a single point of failure.

Although the problem of permanent blocking of resources is solved through the timeout mechanism, there is still a problem of data inconsistency in 3PC. When the participant receives the PreCommit message, if the network is partitioned, the coordinator and the participant cannot communicate normally. In this case, the participant will still commit the transaction.

After understanding 2PC and 3PC, we can see that neither of them can completely solve the problem of data consistency in distributed environment.

JDBC operates MySQL XA transactions

MySQL supports XA distributed transactions since 5.0.3 and is supported only by the InnoDB storage engine. MySQL Connector/J has provided direct support for XA since version 5.0.0.

In the DTP model, MySQL belongs to RM resource manager, so the statement that MySQL supports XA transactions will not be demonstrated here, because it only executes its own single transaction branch. We will demonstrate how to control multiple RM to complete 2PC distributed transactions through TM through JDBC.

Let's start with the need to introduce the Maven version of GAV, because the high version 8.x removes support for XA distributed transactions (probably because no one will use it).

Mysql mysql-connector-java 5.1.38

Here, in order to ensure the comfort of reading on the official account, multiple lines of code are merged into one line through IDEA. If your partner needs to paste it into IDEA, just format it.

Because the XA protocol is based on the 2PC consistency algorithm, partners can understand, simulate errors and execute results against the DTP model and 2PC described in the above article when looking at the code.

Import com.mysql.jdbc.jdbc2.optional.MysqlXAConnection;import com.mysql.jdbc.jdbc2.optional.MysqlXid;import javax.sql.XAConnection;import javax.transaction.xa.XAException;import javax.transaction.xa.XAResource;import javax.transaction.xa.Xid;import java.sql.*; public class MysqlXAConnectionTest {public static void main (String [] args) throws SQLException {/ / true means to print a XA statement for debugging boolean logXaCommands = true / / obtain resource manager operation interface instance RM1 Connection conn1 = DriverManager.getConnection ("jdbc:mysql://localhost:3306/test", "root", "root"); XAConnection xaConn1 = new MysqlXAConnection ((com.mysql.jdbc.Connection) conn1, logXaCommands); XAResource rm1 = xaConn1.getXAResource () / / obtain resource manager operation interface instance RM2 Connection conn2 = DriverManager.getConnection ("jdbc:mysql://localhost:3306/test", "root", "root"); XAConnection xaConn2 = new MysqlXAConnection ((com.mysql.jdbc.Connection) conn2, logXaCommands); XAResource rm2 = xaConn2.getXAResource () / / AP (application) requests TM (transaction manager) to execute a distributed transaction. TM generates a global transaction ID byte [] gtrid = "distributed_transaction_id_1" .getBytes (); int formatId = 1 Try {/ / = execute transaction branch on RM1 and RM2 respectively = / / TM generates transaction branch on RM1 ID byte [] bqual1 = "transaction_001" .getBytes (); Xid xid1 = new MysqlXid (gtrid, bqual1, formatId); / / execute transaction branch rm1.start (xid1, XAResource.TMNOFLAGS) on RM1 PreparedStatement ps1 = conn1.prepareStatement ("INSERT into user (name) VALUES ('jack')"); ps1.execute (); rm1.end (xid1, XAResource.TMSUCCESS); / / TM generates transaction branch ID byte on RM2 [] bqual2 = "transaction_002" .getBytes (); Xid xid2 = new MysqlXid (gtrid, bqual2, formatId); / / execute transaction branch rm2.start (xid2, XAResource.TMNOFLAGS) on RM2 PreparedStatement ps2 = conn2.prepareStatement ("INSERT into user (name) VALUES ('rose')"); ps2.execute (); rm2.end (xid2, XAResource.TMSUCCESS); / / = two-phase commit = / / phase1: ask all RM to prepare commit transaction branch int rm1_prepare = rm1.prepare (xid1); int rm2_prepare = rm2.prepare (xid2) / / phase2: commit all transaction branches if (rm1_prepare = = XAResource.XA_OK & & rm2_prepare = = XAResource.XA_OK) {/ / all transaction branches prepare successfully, commit all transaction branches rm1.commit (xid1, false); rm2.commit (xid2, false) } else {/ / if a transaction branch is unsuccessful, roll back rm1.rollback (xid1); rm1.rollback (xid2);}} catch (XAException e) {e.printStackTrace () At this point, I believe you have a deeper understanding of "how to use the distributed transaction 2PC, 3PC model". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.