Shulou(Shulou.com)06/03 Report--

Web development of distributed transactions is like, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

Transaction: generally refers to what is to be done or done, consisting of all the operations performed between the begin transaction and the end of the transaction (end transaction).

To put it simply: either all are executed or all fail.

That distributed transaction, which is naturally a transaction running in a distributed system, is composed of transactions on many different machines. As above, only if all transactions in the distributed system are executed can it be successful, otherwise it will fail.

Basic characteristics of transactions ACID:

Atomicity: a transaction is an indivisible unit of work, and all the operations included in the transaction are either done or not done.

Consistency: the data is complete before and after the transaction is executed.

Isolation: the execution of one transaction cannot be interfered with by other transactions. That is, the operations and the data used within a transaction are isolated from other concurrent transactions, and the concurrent transactions can not interfere with each other.

Persistence: also known as persistence, once a transaction is committed, its changes to the data in the database should be permanently saved.

Second, the objectives and practical application scenarios of distributed transactions

The goal of distributed transactions: to solve the problem of consistency of multiple independent transactions.

For example, we have a function, the order system, which spans multiple microservices. Because each microservice is not in a library, we cannot use database transactions to guarantee transactions, so we can use distributed transactions at this time.

For example, in the mall project, a user pays an order, and in the payment system, the payment table is updated, and in another order system, the status of the order in the order library becomes paid, so in the order table and the payment table, how can they ensure the transaction between the two databases in different databases?

Payment operation: pay to modify the balance, modify the order status

Third, distributed transaction solution

Two-phase commit protocol (2PC)

Three-phase commit protocol (3PC)

Compensation transaction (TCC)

Implementation of message middleware

Seata framework

IV. Phase II submission Protocol (2PC)

Based on XA protocol, adopt strong consistency and comply with ACID

2PC: (2-phase commit protocol), is based on the XA/JTA specification.

4.1 XA

XA is a distributed transaction architecture (or protocol) proposed by the X/Open organization. The XA architecture mainly defines the interface between (global) transaction manager (Transaction Manager) and (local) resource manager (Resource Manager).

The XA interface is a bi-directional system interface that forms a communication bridge between the transaction manager (Transaction Manager) and one or more resource managers (Resource Manager). In other words, in a transaction based on XA, we can manage multiple resources, such as a system accessing multiple databases, or accessing both databases and resources such as message middleware. In this way, we can directly implement fully committed or cancelled transactions in multiple databases and message middleware. The XA specification is not a java specification, but a general specification.

4.2 JTA

JTA (Java Transaction API), which is the programming interface specification of J2EE, is the JAVA implementation of XA protocol. It mainly defines:

A transaction manager interface javax.transaction.TransactionManager that defines operations related to transaction start, commit, recall, and so on. A resource definition interface javax.transaction.xa.XAResource that meets the XA specification. If a resource wants to support JTA transactions, it needs to have its resources implement the XAResource interface and implement the two-phase commit-related interfaces defined by the interface.

4.3 flow chart

4.4 submission process

1. The request phase, (commit-request phase, or voting phase, voting phase), steps (1-5) during the request phase, the coordinator will inform the transaction participant to prepare to commit or cancel the transaction, and then proceed to the voting process. During the voting process, the participant will inform the coordinator of his own decision: agree (the transaction participant local job executed successfully) or cancel (local job execution failure).

two。 Submission phase (commit phase), steps (6-7) at this stage, the coordinator will make a decision based on the voting results of the first stage: submit or cancel. If and only if all participants agree to commit the transaction coordinator will notify all participants to commit the transaction, otherwise the coordinator will notify all participants to cancel the transaction. After receiving a message from the coordinator, the participant will perform the response operation.

4.5 shortcomings

Single point of failure: the initiation, submission or cancellation of the transaction is managed by the boss coordinator. As long as the coordinator goes down, it will be cool.

The disadvantage of synchronous blocking: as can be seen from the above introduction and examples, when our participating system does not receive the boss's real commit or cancel transaction instructions, it locks the current resources and does not really do some transaction-related operations, so, the whole distributed system environment is blocked.

The disadvantage of data inconsistency: that is to say, when the boss coordinator sends the real commit transaction to the boys, part of the network fails, resulting in part of the system not receiving the real instructions, then part of the submission part is not committed, therefore, this will lead to data inconsistency.

4.6 unsolvable problems

When the coordinator goes wrong and the participant goes wrong, the integrity of the transaction execution cannot be guaranteed in the two phases. Consider that the coordinator goes down after sending the commit message, and the only participant who receives the message is also down. Then even if there is a new coordinator, the status of the transaction is uncertain, and no one knows whether the transaction has been committed. Those who know have been silenced.

Fifth, three-phase submission agreement (3PC)

Adopt strong consistency and comply with ACID. Added in the second phase: timeout and pre-commit mechanism. There are three main stages, canCommit, preCommit and doCommit.

5.1 flowchart

5.2 proc

1.CanCommit phase: the CanCommit phase of 3PC is actually very similar to the preparation phase of 2PC. The coordinator sends a commit request to the participant, and the participant returns the Yes response if it can be submitted, otherwise the No response is returned.

2.PreCommit phase: Coordinator decides whether or not to continue the PreCommit operation of the transaction based on the reaction of Cohort.

Depending on the response, there are two possibilities. a. If the feedback Coordinator gets from all Cohort is an Yes response, then the transaction is pre-executed: a pre-commit request is sent. Coordinator sends a PreCommit request to Cohort and enters the Prepared phase. Transaction pre-commit. When Cohort (a group of soldiers) receives the PreCommit request, it performs a transaction operation and records the undo and redo information in the transaction log. Respond to feedback. If the Cohort successfully executes the transaction operation, the ACK response is returned and the final instruction begins to wait.

b. If any Cohort sends an No response to the Coordinator, or if the Coordinator does not receive a response from the Cohort after the wait timeout, the transaction is interrupted: an interrupt request is sent. Coordinator sends abort requests to all Cohort. Interrupt the transaction. After Cohort receives an abort request from Coordinator (or still does not receive a request from Cohort after a timeout), it executes an interruption of the transaction.

3.DoCommit phase: this phase makes a real transaction commit, which can also be divided into the following two situations:

Execute the submission A. Send a submission request. If Coordinator receives the ACK response from Cohort, he will move from the pre-commit state to the commit state. And send doCommit requests to all Cohort. b. Transaction commit. After the Cohort receives the doCommit request, it performs a formal transaction commit. And release all transaction resources after the transaction commit is completed. c. Respond to feedback. After the transaction is committed, an ACK response is sent to Coordinator. d. Complete the transaction. After Coordinator receives all the ACK responses from Cohort, it completes the transaction.

If the interrupt transaction coordinator does not receive an ACK response from the participant, the interrupt transaction is executed.

a. Send interrupt request coordinator sends abort request B. After receiving the abort request, the transaction rollback participant uses the undo information recorded in phase 2 to perform the transaction rollback operation, and releases all transaction resources after the rollback is completed. c. Feedback result after completing the transaction rollback, the participant sends the ACK message D. After receiving the ACK message from the participant, the interrupt transaction coordinator executes the interruption of the transaction.

5.3 shortcomings

If the Coordinator sends an abort request after entering the PreCommit, it is assumed that only one Cohort has received and performed the abort operation, while other Cohort which is unknown to the system status will choose to continue the Commit according to the 3PC, and the system state will be inconsistent.

5.4 the difference between 2PC and 3PC

Add inquiries to increase the probability of success.

A timeout mechanism is set for both the coordinator (Coordinator) and the participant (Cohort) (in 2PC, only the coordinator has the timeout mechanism, that is, it fails by default if a message from cohort is not received within a certain period of time). The coordinator hung up, and after waiting for the timeout, the participant committed the transaction by default. There is a loss of progress.

If the participant is abnormal and the coordinator is abnormal, it will cause other participants to submit.

Between the preparation phase and the commit phase of the 2PC, insert the pre-commit phase so that the 3PC has three phases: CanCommit, PreCommit and DoCommit. The PreCommit is a buffer that ensures that the state of the participating nodes is consistent before the final commit phase.

VI. The final consistency form based on messages.

Take the final consistency and follow the BASE theory.

BASE: the full name is an acronym for three phrases: Basically Avaliable (basic available), Soft state (soft state), and Eventually consistent (final consistency), proposed by the architect of eBay.

Basically Avaliable: in a distributed system environment, it is allowed to sacrifice the unavailability of some functions that do not affect the main process and downgrade it to ensure the normal availability of core services.

Soft state: it means that in a transaction, we allow the system to have an intermediate state without affecting our system. Take the master-slave replication of the database, for example, there is a delay when replication is completely allowed.

Eventually consistent: take database master-slave replication as an example. Although there is a small delay in master-slave replication, the data will soon be consistent.

Distributed transactions cannot be solved 100% and can only improve the probability of success. Time between two phases, millisecond level. Remedial measure: scheduled task compensation. Program or script compensation. Manual intervention.

7. TCC

Solution: TCC (Try, Confirm, Cancel), two-stage compensation scheme.

As can be seen from the name, to implement a transaction, you need to define three API: pre-possession of resources, confirmation of the submission of actual resources, and unpossession = rollback.

If half of the last two steps fail, log, compensate, and notify the labor.

2PC: a distributed transaction at the resource level that holds locks on resources all the time.

If you lock so many databases across more than a dozen libraries, it will lead to an extreme waste of resources. Reduced throughput.

TCC: distributed transactions at the business level, ultimately consistent, do not always hold locks. The granularity of the lock is reduced, and each time a library is operated, the lock is released.

All are relative: if there is only one request per day, using 2PC is better than TCC. Because tcc has more than one interface call. At this time, the 2PC is not afraid to take up resources, anyway, it is only a call. TCC has more advantages in high concurrency scenarios.

8. Implementation of message middleware

Message queuing flexible transaction flowchart:

1. Operate the payment table, and then insert a piece of data in the event table with the status of new, and put it into the database. This (1, 2, 3) operation is all in a transaction, because they are all a library.

2. The scheduled task reads the event table and sends the queue. After the sending is successful, change the status of the event table new to (published), listen to the event table, and insert a piece of data into the event table.

3. Is the scheduled task reading library a published event table? if it is a published event table, update the order table and update the event table to processed, so that a large transaction is divided into several small transactions.

Table design:

CREATE TABLE `tnormalorderEvent` (`id`int (16) NOT NULL, `order_ type` varchar (32) DEFAULT NULL COMMENT 'event type (payment completed, order table modification status)', `process`varchar (32) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci DEFAULT NULL COMMENT 'event link (new,published,processed)', `content`varchar 'event content Save the data to be passed when the event occurs', `create_ time` datetime DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP, `update_ time`datetime DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`id`) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci

IX. Seata framework

Seata is an open source distributed transaction solution dedicated to providing high-performance and easy-to-use distributed transaction services. Seata will provide users with AT, TCC, SAGA and XA transaction modes to create an one-stop distributed solution for users.

Official website Api (highly recommended): https://seata.io/zh-cn/docs/overview/what-is-seata.html

Seata download address: https://seata.io/zh-cn/blog/download.html

Flow chart:

Procedure:

1. Download seata server

Https://seata.io/zh-cn/blog/download.html

2. Modify file.conf

Service {# transaction service group mapping # modify, but do not change, my_test_tx_group name casually. Vgroup_mapping.my_test_tx_group = "default" # only support when registry.type=file Please don't set multiple addresses # address of this service default.grouplist = "127.0.0.1 store location dir dir 8091" # disable seata disableGlobalTransaction = false} store {# # store mode: file, db # modify mode = "db" # file store property file {# # store location dir dir = "sessionStore"} # database store property # db Information Modification db {# the implement of javax.sql.DataSource Such as DruidDataSource (druid) / BasicDataSource (dbcp) etc. Datasource = "druid" # # mysql/oracle/h3/oceanbase etc. Db-type = "mysql" driver-class-name = "com.mysql.cj.jdbc.Driver" url = "jdbc:mysql://127.0.0.1:3306/seata-server?useUnicode=true&useSSL=false&characterEncoding=utf8&serverTimezone=Asia/Shanghai" user = "root" password = "root"}}

3. Modify registry.conf

Registry {# file 、 nacos 、 eureka 、 redis 、 zk 、 consul 、 etcd3 、 Sofa # modify type = "eureka" nacos {serverAddr = "localhost" namespace = "cluster =" default "} # modify eureka {serviceUrl =" http://localhost:8761/eureka" application = "default" weight = "1"} redis {serverAddr = "localhost:6379" db = "0"} zk {cluster = "default" serverAddr = "127.0.0" .1connect.timeout 2181 "session.timeout = 6000 connect.timeout = 2000} consul {cluster =" default "serverAddr =" 127.0.0.1VR 8500 "} etcd3 {cluster =" default "serverAddr =" http://localhost:2379"} sofa {serverAddr = "127.0.0.1 Vol 9603" application = "default" region = "DEFAULT_ZONE" datacenter = "DefaultDataCenter" cluster = "default" group = "SEATA_GROUP" addressWaitTime = "3000"} file {name = "file.conf"} config {# file, Nacos 、 apollo 、 zk 、 consul 、 Etcd3 type = "file" nacos {serverAddr = "localhost" namespace = ""} consul {serverAddr = "127.0.0.1 seata-server 8500"} apollo {app.id = "seata-server" apollo.meta = "http://192.168.1.204:8801"} zk {serverAddr =" 127.0.0.1 nacos 2181 "session.timeout = 6000 connect.timeout = 2000} Etcd3 {serverAddr = "http://localhost:2379"} file {name =" file.conf "}}

4. Create a database and a table

Branch transaction table: branchtable global transaction table: globaltable global lock: lock_table

Note: the structure of the table cannot be wrong.

5. Add undo_log to each library for rollback

CREATE TABLE `undo_ log` (`id`bigint (20) NOT NULL AUTO_INCREMENT, `branch_ id` bigint (20) NOT NULL, `xid` varchar (100th) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL, `context`varchar (128) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL, `rollback_ info` longblob NOT NULL, `log_ status` int (11) NOT NULL, `log_ created` datetime NOT NULL, `log_ modified` datetime NOT NULL, `ext` varchar (100th) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL, PRIMARY KEY (`id`) USING BTREE UNIQUE KEY `ux_undo_ log` (`xid`, `branch_ id`) USING BTREE) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.