Shulou(Shulou.com)05/31 Report--

This article shows you what the distributed transaction solution under the micro-service architecture is like, the content is concise and easy to understand, it can definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

1 the development of micro services

Microservices advocate splitting complex single applications into several services with simple functions and loose coupling, which can reduce the difficulty of development, enhance scalability and facilitate agile development. At present, it is respected by more and more developers, and many Internet industry giants and open source communities have begun the discussion and practice of micro-services. Hailo consists of 160 different services, while NetFlix has about 600 services. Domestically, Alibaba, Tencent, JD.com, 58.com and many other Internet companies have carried out micro-service practice. At present, there are many development frameworks of micro-services, such as Dubbo, SpringCloud, thrift, grpc and so on.

2 problems existing in the landing of micro-services

Although micro-service is in full swing, its practice is still in the exploratory stage. In view of experience, technical strength and other problems, it is difficult for many small and medium-sized Internet companies to land micro-services. As the famous architect Chris Richardson said, the main difficulties are as follows:

The main results are as follows: 1) after the single application is divided into distributed systems, the communication mechanism and fault handling measures between processes become more complex.

2) after the micro-service of the system, a seemingly simple function may need to invoke multiple services and operate multiple database implementations, and the distributed transaction problem of service invocation becomes very prominent.

3) there are a large number of micro-services, which makes it more difficult to test, deploy and monitor them.

With the maturity of the RPC framework, the first problem has been gradually solved. For example, dubbo can support a variety of communication protocols, springcloud can very well support restful calls. For the third question, with the development of docker and devops technology and the introduction of automated operation and maintenance tools for public cloud paas platforms, the testing, deployment and operation and maintenance of micro services will become easier and easier.

As for the second problem, there is no general solution to solve the transaction problems caused by micro-services. Distributed transaction has become the biggest obstacle to the landing of micro-services, and it is also one of the most challenging technical problems. For this reason, this paper will deeply discuss the various solutions of distributed transactions under the micro-service architecture, and focus on interpreting the distributed transaction solution proposed by Alibaba-GTS. GTS mentioned in this scheme is a new generation of distributed transactional Internet middleware to solve micro-service problems.

3 SOA distributed transaction solution 3.1 two-phase commit scheme based on XA protocol

Transaction middleware and database use two-phase commit to complete a global transaction through XA interface specification. XA specification is based on two-phase commit protocol.

The first stage is the voting stage, in which all participants send feedback on the success of the transaction to the coordinator; the second stage is the implementation phase, in which the coordinator notifies all participants based on the feedback of all participants. submit or roll back on all branches in step. As shown in the following figure:

The two-phase commit scheme is widely used, and almost all commercial OLTP databases support XA protocol. However, the two-phase commit scheme takes a long time to lock resources, which has a great impact on performance, and is basically not suitable to solve the problem of micro-service transactions.

3.2 TCC scenario

There are many TCC schemes in e-commerce and financial fields. The TCC scheme is actually an improvement of two-phase submission. It explicitly divides each branch of the entire business logic into three operations: Try, Confirm and Cancel. The Try part completes the preparation of the business, the confirm part completes the business commit, and the cancel part completes the rollback of the transaction. The basic principles are shown in the following figure:

At the beginning of the transaction, the business application registers the startup transaction with the transaction coordinator. The business application then invokes the try interfaces of all services to complete the first phase of preparation. The transaction coordinator then decides to call the confirm interface or the cancel interface based on the return of the try interface. If the interface call fails, a retry will be made.

The TCC scheme allows applications to define the granularity of database operations, which makes it possible to reduce lock conflicts and improve throughput. Of course, the TCC proposal also has its shortcomings, which are mainly reflected in the following two aspects:

It is highly intrusive to the application. Each branch of business logic needs to implement three operations: try, confirm and cancel. The application is intrusive and the transformation cost is high.

It is difficult to realize. Different rollback strategies need to be implemented according to different failure reasons, such as network status, system failure and so on. In order to meet the requirements of consistency, confirm and cancel interfaces must be idempotent.

As a result of the above reasons, TCC programs are mostly adopted by large companies with strong R & D strength and urgent needs. Micro-service advocates lightweight and easy deployment of service, while the processing logic of many transactions in TCC scheme needs to be implemented by self-coding, which is complex and has a large amount of development.

3.3 final consistency scheme based on messages

The message consistency scheme is to ensure the consistency of upstream and downstream application data operations through message middleware. The basic idea is to put the local operation and sending messages in a single transaction, ensuring that either the local operation and message delivery succeed or fail. The downstream application subscribes the message to the message system and performs the corresponding operation after receiving the message. As shown in the following figure:

In essence, the message scheme transforms the distributed transaction into two local transactions, and then relies on the retry mechanism of the downstream business to achieve the final consistency. The final consistency scheme based on message is also very intrusive to the application, which requires a lot of business transformation and the cost is high.

4 GTS- distributed transaction solution

GTS is a distributed transaction middleware developed by Alibaba middleware department, which can provide one-stop solution for distributed transactions in micro-service architecture.

For more GTS information, please visit the R & D team Weibo.

4.1Core benefits of GTS

Super performance

Through a lot of innovation, GTS has solved the problems of transaction ACID characteristics and high performance, high availability, low intrusion. The average response time of a single transaction branch is about 2ms, and a cluster of three servers can support more than 30, 000 TPS of distributed transaction requests.

Application invasiveness is extremely low

GTS has a low intrusion into the business, and the business code only needs to add at least one line of annotation (@ TxcTransaction) to declare the transaction. The separation of business and transaction liberates micro-services from transactions, micro-services focus on the business itself, and no longer need to consider complex issues such as reverse interface, idempotent, rollback strategy, which greatly reduces the difficulty and workload of micro-service development.

Complete solution

GTS supports a variety of mainstream service frameworks, including EDAS,Dubbo,Spring Cloud.

In some cases, the application needs to call the interface of the third-party system, and the third-party system is not connected to the GTS. At this point, you need to use GTS's MT mode. The MT pattern of GTS can be equivalent to the TCC pattern, and users can customize the specific behavior of each transaction stage according to their own business needs. MT mode provides more flexibility and possibility to achieve custom optimization and special functions in special scenarios.

Strong fault tolerance

GTS solves the single point problem of XA transaction coordinator, achieves true high availability, and ensures strict data consistency in all kinds of exception cases.

4.2 Application scenarios for GTS

GTS can be used in many areas involving service invocation, including but not limited to financial payment, telecommunications, e-commerce, express logistics, advertising marketing, social networking, instant messaging, mobile games, video, Internet of things, car Internet of things, etc. For a detailed introduction, you can read the article "GTS- Alibaba New distributed transaction solution".

4.3Integration of GTS and micro services

GTS includes three parts: client (GTS Client), resource manager (GTS RM) and transaction coordinator (GTS Server). GTS Client is mainly used to define the boundary of the transaction and complete the initiation and termination of the transaction. GTS RM completes the creation, commit, rollback and other operations of transaction branches. GTS Server is mainly responsible for the overall promotion of distributed transactions and the management of transaction life cycle. The structure diagram of GTS and micro-service integration is shown below. GTS Client needs to be integrated with business applications, and RM and micro-services integrated deployment.

4.4 output form of GTS

GTS currently has three output forms: public cloud output, public network output, and proprietary cloud output.

4.4.1 Public Cloud output

This form of output is for Aliyun users. If your business system has been deployed to Aliyun, you can apply to activate a public cloud GTS. After activation, business applications can guarantee the consistency of service invocation through GTS. In this usage scenario, the network environment between the business system and GTS is ideal and achieves good performance.

4.4.2 Public network output

This output format is more convenient and flexible for non-Aliyun users. As long as the business system can connect to the Internet, it can enjoy the cloud services provided by GTS (the difference with public cloud output is that the client is deployed locally rather than on the cloud).

In the normal network environment, take the global transaction containing two local transactions as an example, the transaction completion time is about 20ms, 50 concurrency can easily achieve distributed transactions above 1000TPS, which is sufficient for the vast majority of businesses. In the public network environment, network flash is difficult to avoid completely. In this case, GTS can still ensure the data consistency of service invocation.

For specific use examples, see the project samples in Section 4.7 GTS.

4.4.3 proprietary cloud output

This form is mainly aimed at large users who have built their own proprietary cloud platform. GTS can be deployed directly to users' proprietary cloud to provide distributed transaction services for proprietary cloud. At present, more than 10 private clouds of super-large enterprises have used GTS to solve distributed transaction problems, and their performance and stability have been strictly tested by users.

4.5 ways to use GTS

GTS is very intrusive to applications and easy to use. The following is an example of an order storage application. The order business application completes the order business by calling the order service and inventory service, and the service development framework is Dubbo.

4.5.1 order business application

Use the @ TxcTransaction annotation on the periphery of the business function to start distributed transactions. The Dubbo application propagates the transactional xid of GTS to the server by hiding parameters.

@ TxcTransaction (timeout = 1000 * 10) public void Bussiness (OrderService orderService, StockService stockService, String userId) {/ / get transaction context String xid = TxcContext.getCurrentXid (); / / pass xid to a server via RpcContext RpcContext.getContext (). SetAttachment ("xid", xid); / / execute your own business logic int productId = new Random (). NextInt (100); int productNum = new Random (). NextInt (100) OrderDO orderDO = new OrderDO (userId, productId, productNum, new Timestamp (new Date (). GetTime ()); orderService.createOrder (orderDO); / / transfer xid to another server via RpcContext RpcContext.getContext (). SetAttachment ("xid", xid); stockService.updateStock (orderDO);} 4.5.2 service provider

Update inventory method

Public int updateStock (OrderDO orderDO) {/ / get the global transaction ID and bind to the context String xid = RpcContext.getContext () .getAttachment ("xid"); TxcContext.bind (xid,null); / / execute your own business logic int ret = jdbcTemplate.update ("update stock set amount = amount -? Where product_id =? ", new Object [] {orderDO.getNumber (), orderDO.getProductId ()}); TxcContext.unbind (); return ret;} 4.6 GTS

GTS has been widely used in Taobao, Tmall, Ali Pictures, Taopiaopiao, Alimama, 1688 and other Ali business systems, and has withstood the test of massive requests for Singles' Day in 16 and 17 years. The highest traffic of an online business system has reached 100, 000 TPS (100000 transactions per second).

After the output of public cloud and private cloud, GTS has more than 100 online users, and many users solve the distributed transaction problems of service frameworks such as SpringCloud, Dubbo, Edas and so on through GTS. Business areas involving power, logistics, ETC, tobacco, finance, retail, e-commerce, shared travel and other industries, have been unanimously recognized by users.

The picture above shows the integration of GTS and SpringCloud, which is applied to a shared travel system. In the business sharing travel scenario, GTS supports the data consistency of various applications such as Internet of things system, order system, payment system, operation and maintenance system, analysis system, etc., to ensure massive orders and tens of millions of pipelined transactions.

4.7Engineering example of GTS

For public cloud samples of GTS, please refer to Aliyun's website. Two sample projects of sample-txc-simple and sample-txc-dubbo are provided in the public network environment.

4.7.1 sample-txc-simple sample 4.7.1.1 sample business logic

This sample is the entry sample for GTS, and the business logic of the case is to transfer money from account A to account B, where An and B are located in two MySQL databases, respectively, and use GTS transactions to ensure that the total amount of money in accounts An and B remains the same.

4.7.1.2 sample building method

1) prepare the database environment

Install MySQL and create two databases, db1 and db2. Create txc_undo_ log tables in db1 and db2 respectively (see 4.7.3 for the SQL script). Create the user_money_a table in the db1 library and the user_money_ b table in the db2 library.

2) download sample

Download the sample-txc-simple file locally, and the SDK of GTS is already included in the sample.

3) modify the configuration

Open txc-client-context.xml under the sample-txc-simple/src/main/resources directory and change the url, username and password of the data source to the actual values.

4) running sample

Execute the build.sh compilation project under the sample-txc-simple directory. Execute run.sh when the compilation is complete.

4.7.2 sample-txc-dubbo sample 4.7.2.1 sample business logic

This case simulates the business logic of placing orders and reducing inventory. The client (Client) creates the order by calling the order service (OrderService) and then deducts the inventory by calling the inventory service (StockService). The order service reads and writes the order database, and the inventory service reads and writes the inventory database. Consistency of cross-service transactions is guaranteed by GTS.

4.7.2.2 sample building method

1) prepare the database environment

Install MySQL and create two databases, db1 and db2. Create txc_undo_ log tables in db1 and db2, respectively. Create the orders table in the db1 library and the stock table in the db2 library.

2) download sample

Download the sample file sample-txc-dubbo to your local machine, and the SDK of GTS is already included in the sample.

3) modify the configuration

Open the sample-txc-dubbo/src/main/resources directory and change the url, username and password of the data sources in the dubbo-order-service.xml and dubbo-stock-service.xml files to the actual values.

4) running sample

a. Compiler

Execute the build.sh command in the project root directory to compile the project. After compilation, three running scripts, order_run.sh, stock_run.sh and client_run.sh, are generated in the sample-txc-dubbo/client/bin directory corresponding to order service, inventory service and client.

b. Run the program

Execute run.sh in the root directory, and the script starts order_run.sh (order service), stock_run.sh (inventory service), and client_run.sh (client program) in turn.

4.7.2.3 other instructions

The sample uses the declaration method of the Multicast registry. If the local machine uses a wireless network, it is possible for the dubbo service to obtain the ipv6 address when binding the address, which can be disabled through the jvm startup parameter.

The method is to configure the jvm startup parameter-Djava.net.preferIPv4Stack=true.

4.7.3 SQL4.7.3.1 table creation txc_undo_log

CREATE TABLE txc_undo_log (

Id bigint (20) NOT NULL AUTO_INCREMENT COMMENT 'primary key'

Gmt_create datetime NOT NULL COMMENT 'creation time'

Gmt_modified datetime NOT NULL COMMENT 'modification time'

Xid varchar (100) NOT NULL COMMENT 'Global transaction ID'

Branch_id bigint (20) NOT NULL COMMENT 'branch transaction ID'

Rollback_info longblob NOT NULL COMMENT 'LOG'

Status int (11) NOT NULL COMMENT 'status'

DB IP' where the server varchar (32) NOT NULL COMMENT 'branch is located

PRIMARY KEY (id)

KEY unionkey (xid,branch_id)

) ENGINE=InnoDB AUTO_INCREMENT=211225994 DEFAULT CHARSET=utf8 COMMENT=' transaction log table'

4.7.3.2 create a table user_money_a

CREATE TABLE user_money_a (

Id int (11) NOT NULL AUTO_INCREMENT

Money int (11) DEFAULT NULL

PRIMARY KEY (id)

) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8

4.7.3.3 create a table user_money_b

CREATE TABLE user_money_b (

Id int (11) NOT NULL AUTO_INCREMENT

Money int (11) DEFAULT NULL

PRIMARY KEY (id)

) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8

4.7.3.4 create a table orders

CREATE TABLE orders (

Id bigint (20) NOT NULL AUTO_INCREMENT

User_id varchar (255) NOT NULL

Product_id int (11) NOT NULL

Number int (11) NOT NULL

Gmt_create timestamp NOT NULL

PRIMARY KEY (id)

) ENGINE=MyISAM AUTO_INCREMENT=351 DEFAULT CHARSET=utf8

4.7.3.5 create a table stock

CREATE TABLE stock (

Product_id int (11) NOT NULL

Price float NOT NULL

Amount int (11) NOT NULL

PRIMARY KEY (product_id)

) ENGINE=InnoDB DEFAULT CHARSET=utf8

The above is what the distributed transaction solution is like under the micro-service architecture. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.