Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "what are the solutions for distributed transactions". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Generation of distributed transactions

Let's first look at the definition of distributed transaction on Baidu: distributed transaction means that the participants of the transaction, the server that supports the transaction, the resource server and the transaction manager are located on different nodes of different distributed systems.

Er ~ a little abstract, simply draw a picture to understand, take the single reduction of inventory, deduction balance as an example:

When the volume of the system is very small, the single architecture can fully meet the existing business needs, all businesses share a database, and the whole order issuing process may only be used to operate the database under the same transaction in one method. At this point, it is easy to either commit all or roll back all of the operations.

Sub-library and sub-table, SOA

However, with the continuous growth of business, the single architecture gradually can not bear the huge traffic, so it is necessary to separate the database and table, and split the application of SOA service. It also produces the order center, user center, inventory center and so on, which brings the problem that the business is isolated from each other, each business maintains its own database, and the data exchange can only be called RPC.

When users place an order again, they need to operate on the order library order, repository storage and user library account at the same time. At this time, we can only guarantee our local data consistency, but not the success of calling other services. Therefore, in order to ensure the data consistency of the entire order issuing process, we need distributed transaction intervention.

Seata advantage

There are many solutions to implement distributed transactions, such as 2PC and 3PC based on XA protocol, TCC based on business layer, final consistency scheme based on message queuing and message table, and Seata middleware to be discussed today. Let's take a look at the advantages and disadvantages of each scheme.

2PC

For distributed transactions based on XA protocol, XA protocol is divided into two parts: transaction manager and local resource manager. Among them, the local resource manager is often implemented by databases, such as Oracle and MYSQL, which implement the XA interface, while the transaction manager acts as a global scheduler.

Two-phase commit (2PC) has little business intrusions, and its most important advantage is to make clients transparent. Users can make distributed transactions based on XA protocol just like local transactions, which can strictly guarantee the ACID characteristics of transactions.

But the disadvantage of 2PC is also obvious, it is a highly consistent synchronous blocking protocol, the transaction execution process needs to lock all the required resources, which is commonly known as rigid transactions. Therefore, it is more suitable for short transactions determined by the holding time, and the overall performance is relatively poor.

Once the transaction coordinator goes down or network jitter occurs, it will keep the participants in the state of locking resources or only some participants submit successfully, resulting in data inconsistency. Therefore, distributed transactions based on XA protocol are not the best choice in the scenario of concurrent performance.

3PC

Three-stage commit (3PC) is an improved version of two-phase commit (2PC). In order to solve the blocking problem of the two-phase commit protocol, two-stage commit is mentioned above. When the coordinator crashes, the participant cannot make the final choice and will keep blocking and locking resources.

In 2PC, only the coordinator has a timeout mechanism, and 3PC introduces a timeout mechanism in both the coordinator and the participants. after the coordinator fails, the participants will not block all the time. Moreover, a preparatory phase is inserted in the first and second stages (as shown in the following figure, which looks a little verbose) to ensure that the state of the participating nodes is consistent before the final commit phase.

Although 3PC uses timeout mechanism to solve the blocking problem of participants after coordinator failure, but at the same time, there is one more network communication, which makes the performance worse and less recommended.

TCC

The so-called TCC programming model is also a variation of two-phase commit, except that TCC implements two-phase commit for writing code at the business layer. TCC refers to Try, Confirm and Cancel, respectively. A business operation should be written to these three methods.

The following single deduction inventory as an example, the inventory is occupied in the Try phase, and the inventory is actually withheld in the Confirm phase. If the inventory deduction fails, the inventory is rolled back in the Cancel phase to release the inventory.

TCC does not have the problem of resource blocking, because each method commits the transaction directly, and once an exception passes, Cancel rolls back to compensate, which is often called compensatory transaction.

Originally one method, but now need three methods to support, we can see that TCC is very intrusive to the business, and this pattern can not be well reused, which will lead to a surge in development. Also take into account the network fluctuations and other reasons, in order to ensure that requests must be delivered will have a retry mechanism, so take into account the idempotency of the interface.

Message transaction (final consistency)

In fact, message transaction is based on the two-phase commit of message middleware, which puts the local transaction and sending messages in the same transaction to ensure the success of local operation and sending messages at the same time. Order deduction inventory schematic:

The order system sends a pre-deduction inventory message to MQ, and MQ saves the reserve message and returns a successful ACK

Upon receipt of the prepared message, ACK is successfully executed, and the order system executes the local order operation. In order to prevent the message from being sent successfully and the local transaction fails, the order system implements the callback API of MQ, which constantly checks whether the local transaction is executed successfully. If it fails, rollback rolls back the prepared message; if it succeeds, the message is finally submitted by commit.

The inventory system consumes the deduction inventory message and executes the local transaction. If the deduction fails, the message will be reinvested. Once the number of retries is exceeded, the failure message will be persisted on the local table, and the scheduled task will be started to compensate.

The two-phase commit scheme based on message middleware is usually used in high concurrency scenarios, sacrificing the strong consistency of data for a significant improvement in performance, but the cost and complexity of implementing this method is relatively high, depending on the actual business situation.

Seata

Seata is also a distributed transaction solution evolved from two-stage commit, which provides transaction modes such as AT, TCC, SAGA and XA. Here we focus on AT mode.

Since Seata is a two-paragraph submission, let's see what it does at each stage. Below, we also give the following examples of deduction of inventory and balance.

Let's first introduce several roles of Seata distributed transactions:

Transaction Coordinator (TC): a global transaction coordinator that coordinates the state of global and branch transactions (different services) and drives the rollback or commit of global and branch transactions.

Transaction Manager ™: transaction manager, used in the business layer to open / commit / roll back an overall transaction (open a transaction with annotations in the method that invokes the service).

Resource Manager (RM): resource manager, generally means that the business database represents a branch transaction (Branch Transaction). The management branch transaction coordinates with TC to register the branch transaction and report the status of the branch transaction, driving the commit or rollback of the branch transaction.

Seata implements distributed transactions and designs a key role UNDO_LOG (rollback log record table). We create this table in the business library of each application of distributed transactions. The core function of this table is to organize the data mirror of business data before and after updates into rollback logs and back them up in the UNDO_LOG table, so that business exceptions can be rolled back at any time.

The first stage

For example: next we update the name field of the user table.

Update user set name = 'Xiaofu is the most handsome' where name = 'something inside the programmer'

First of all, Seata's JDBC data source agent parses the business SQL to extract the SQL metadata, that is, to get the SQL type (UPDATE), table (user), conditions (where name = 'programmer things') and other related information.

The flow chart of the first stage

First query the data before mirroring, according to the parsed condition information, generate a query statement to locate a piece of data.

Select name from user where name = 'something inside the programmer'

Mirroring before data

Then execute the business SQL to query the mirrored data according to the primary key of the former mirror data.

Select name from user where id = 1

Mirroring after data

The data mirror before and after the update of business data is organized into rollback logs, and the update and rollback logs of business data are submitted in the same local transaction and inserted into the business table and UNDO_LOG table respectively.

The format of rollback record data is as follows: pre-afterImage mirror, post-beforeImage mirror, branchId branch transaction ID, xid global transaction ID

{"branchId": 641789253, "xid": "xid:xxx" "undoItems": [{"afterImage": {"rows": [{"fields": [{"name": "id" "type": 4, "value": 1}]}], "tableName": "product"} "beforeImage": {"rows": [{"fields": [{"name": "id", "type": 4 "value": 1}]}], "tableName": "product"}, "sqlType": "UPDATE"}]}

This ensures that any updates to the submitted business data must have a corresponding rollback log.

Before the local transaction is committed, each branch transaction needs to register the branch (Branch Id) with the global transaction coordinator TC, apply for a global lock for the record to be modified, and use the SELECT FOR UPDATE statement to lock this data. If you can't get the lock all the time, you need to roll back the local transaction. When TM starts a transaction, it generates a globally unique XID, which is passed between the invoked services.

With such a mechanism, the local transaction branch (Branch Transaction) can commit during the first phase of the global transaction and immediately release resources locked by the local transaction. Compared with traditional XA transactions releasing resources in the second stage, Seata reduces the lock scope and improves efficiency. Even if an exception occurs in the second stage and needs to be rolled back, the corresponding rollback data can be quickly found from the UNDO_LOG table and decomposed into SQL to achieve rollback compensation.

Finally, the local transaction is committed, and the update of the business data is submitted together with the previously generated UNDO LOG data, and the result of the local transaction commit is reported to the global transaction coordinator TC.

The second stage

The second stage is to submit or roll back in accordance with the resolutions of each branch:

If the resolution is a global commit, the branch transaction is committed and successful, and the global transaction coordinator (TC) sends a second-phase request to the branch. When a branch submission request from TC is received, the request is placed in an asynchronous task queue and the successful result is immediately returned to TC. The corresponding UNDO LOG rollback records are found and deleted asynchronously and in batches based on Branch ID in the asynchronous queue.

If the resolution is a global rollback, the process is a little more troublesome than global submission. The RM server receives a rollback request from the global coordinator of TC, finds the corresponding rollback log record through XID and Branch ID, and generates a reverse update SQL through the rollback record and executes it to complete the branch rollback.

Note: the rollback logging operation is deleted here, it must be after the local business transaction is executed

The advantages and disadvantages of several kinds of distributed transactions are mentioned above. Let's practice the Seata in the middle of distributed transactions.

Seata practice

Seata is a middleware that needs to be deployed independently, so take Seata Server first. Here, take the latest version of seata-server-1.4.0 as an example, download address: https://seata.io/en-us/blog/download.html

We only need to care about the file.conf and registry.conf files in the\ seata\ conf directory after the decompression.

Seata Server

File.conf

The file.conf file is used to configure the persistent transaction log mode, which currently provides three ways: file, db and redis.

File.conf file configuration

Note: after selecting db method, you need to create three tables in the corresponding database: globalTable (persistent global transaction), branchTable (persistent transaction of each committed branch) and lockTable (persistent locking resource transaction of each branch).

-- the table to store GlobalSession data-persistent global transactions CREATE TABLE IF NOT EXISTS `transaction_service_ Table` (`xid` VARCHAR (1288) NOT NULL, `transaction_ id` BIGINT, `status`TINYINT NOT NULL, `application_ id` VARCHAR (32), `transaction_service_ group`VARCHAR (32), `transaction_ name` VARCHAR (128) `timeout` INT, `time` BIGINT, `application_ data` VARCHAR (2000), `gmt_ create` DATETIME, `gmt_ modified` DATETIME, PRIMARY KEY (`xid`), KEY `idx_gmt_modified_ status` (`gmt_ modified`, `status`), KEY `idx_transaction_ id` (`transaction_ id`) ENGINE = InnoDB DEFAULT CHARSET = utf8 -- the table to store BranchSession data-persistent transaction CREATE TABLE IF NOT EXISTS `VARCHAR table` (`BIGINT NOT NULL id` BIGINT NOT NULL, `xid` VARCHAR), `transaction_ id` BIGINT, `resource_group_ id` VARCHAR (32), `resource_ id` VARCHAR (32), `branch_ type` VARCHAR (8), `status` TINYINT of each commit branch `client_ id` VARCHAR (64), `application_ data` VARCHAR (2000), `gmt_ create` DATETIME (6), `gmt_ modified` DATETIME (6), PRIMARY KEY (`branch_ id`), KEY `idx_ xid` (`xid`) ENGINE = InnoDB DEFAULT CHARSET = utf8 -- the table to store lock data-- persistence of each branch lock table transaction CREATE TABLE IF NOT EXISTS `lock_ table` (`row_ key` VARCHAR) NOT NULL, `xid` VARCHAR (96), `transaction_ id` BIGINT, `branch_ id` BIGINT NOT NULL, `resource_ id` VARCHAR (256), `table_ name` VARCHAR (32), `pk` VARCHAR (36), `gmt_ create` DATETIME `gmt_ modified` DATETIME, PRIMARY KEY (`row_ key`), KEY `idx_branch_ id` (`branch_ id`) ENGINE = InnoDB DEFAULT CHARSET = utf8

Registry.conf

The registry.conf file sets the registry and configuration center:

Currently, the registry supports nacos, eureka, redis, zk, consul, etcd3 and sofa. Here I use eureka as the registry center; the configuration center supports nacos, apollo, zk, consul and etcd3.

Registry.conf file configuration

After configuration, start seata-server in the\ seata\ bin directory, and the server of the Seata will be set up.

Seata Client

After the Seata Server environment is built, we create three new services: order-server (order service), storage-server (inventory deduction service) and account-server (account amount service), which are registered with eureka respectively.

The general core configuration of each service is as follows:

Spring: application: name: storage-server cloud: alibaba: seata: tx-service-group: my_test_tx_group datasource: driver-class-name: com.mysql.jdbc.Driver url: jdbc:mysql://47.93.6.1:3306/seat-storage username: root password: root # eureka registration Heart eureka: client: serviceUrl: defaultZone: http://${eureka.instance.hostname}:8761/eureka/ instance: hostname: 47.93.6.5 prefer-ip-address: true

General business process: the user initiates the order request, the local order order service creates the order record, and calls the storage inventory deduction service and account account balance service remotely through RPC. Only the three services execute successfully at the same time is a complete order issuing process. If one service fails, all other services are rolled back.

Seata is very intrusive to business code, and you only need to open a global transaction with the @ GlobalTransactional annotation in the code.

@ Override @ GlobalTransactional (name = "create-order", rollbackFor = Exception.class) public void create (Order order) {String xid = RootContext.getXID (); LOGGER.info ("- > transaction start"); / / Local method orderDao.create (order); / / remote method inventory deduction storageApi.decrease (order.getProductId (), order.getCount ()) / / remote method deducts account balance LOGGER.info ("- > deduction account start order"); accountApi.decrease (order.getUserId (), order.getMoney ()); LOGGER.info ("- > deduction account end order"); LOGGER.info ("- > transaction end"); LOGGER.info ("Global transaction xid: {}", xid) }

As mentioned earlier, when Seata AT mode implements distributed transactions, a undo_log table must be created in the relevant business library to store the data rollback log. The table structure is as follows:

-- for AT mode you must to init this sql for you business database. The seata server not need it. CREATE TABLE IF NOT EXISTS `undo_ log` (`id` BIGINT (20) NOT NULL AUTO_INCREMENT COMMENT 'increment id', `rollback_ id` BIGINT (20) NOT NULL COMMENT' branch transaction id', `xid` VARCHAR) NOT NULL COMMENT 'global transaction id', `context` VARCHAR (128) NOT NULL COMMENT' undo_log context,such as serialization', `rollback_ info` LONGBLOB NOT NULL COMMENT 'rollback info' `log_ status` INT (11) NOT NULL COMMENT '0:normal status,1:defense status', `log_ created`DATETIME NOT NULL COMMENT' create datetime', `log_ modified`DATETIME NOT NULL COMMENT 'modify datetime', PRIMARY KEY (`id`), UNIQUE KEY `ux_undo_ log` (`xid`, `branch_ id`) ENGINE = InnoDB AUTO_INCREMENT = 1 DEFAULT CHARSET = utf8 COMMENT =' AT transaction mode undo table'

The work of setting up the environment is done. The GitHub address will be posted at the back of the complete case, so it will not take up space here.

Test Seata

The service invocation process in the project is shown in the following figure:

Service invocation process

After starting various services, we directly request the API to issue an order to see the effect. As long as the order order table creates a record, the number of used fields in the storage inventory table and the number of used fields in the account balance table indicates that the order issuing process is successful.

raw data

There is no problem with the forward process after the request, and the data is as expected.

Order data issued

And it is found that the console of the TM transaction manager order-server service also prints out the log of the two-phase commit.

The console submitted twice

So if one of the services is abnormal, will it be rolled back normally? Simulate a timeout exception in the account-server service to see if a global transaction rollback can be achieved.

Global transaction rollback

It is found that none of the data was executed successfully, indicating that the global transaction rollback was also successful.

Take a look at the changes in the undo_log rollback log table. Because Seata deletes the rollback log very quickly, if you want to see the rollback log in the table, you must break a point on a certain service to see it more clearly.

Roll back the record

This is the end of the content of "what are the solutions for distributed transactions"? thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.