Shulou(Shulou.com)06/02 Report--

This article is about how to implement distributed transactions in Spring. I think it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

Implementation principles of distributed system

So how should we implement transactions in a distributed system? This needs to start with the principle of distributed system. There are several kinds of principles for the implementation of distributed systems, such as BASE principle and ACP principle.

Where ACP is:

A: availability (Availability)

C: consistency (Consistency)

P: partition fault tolerance (Tolerance of network Partition)

An and P have nothing to say, but they are the basic characteristics of a distributed system. C (consistency) refers to the consistency of data among multiple nodes in a distributed system, including the data modified by one node. It can also be seen when accessed through another node; and when an operation needs to modify the data of multiple data sources, multiple modifications should be completed or not completed.

The consistency here can be seen as the unity of atomicity, consistency, and even isolation in the ACID features of database transactions mentioned above. If we take the four characteristics of ACID as the requirements to implement a distributed system, it is impossible in reality, in which atomicity can not be realized. If a business request wants to modify the data in multiple databases, then the operation of these multiple databases will not be able to achieve atomicity, and it is bound to be completed on the first database and then on the second database. Then a little time during this period violates atomicity.

Therefore, we are often unable to achieve complete consistency in distributed systems, so we have the BASE theory. BASE is the abbreviation of three phrases: Basically Available (basic available), Soft state (soft state), and Eventually consistent (ultimate consistency). BASE theory is the result of the tradeoff between consistency and usability in CAP, which requires the realization of final consistency.

Among them, Soft state (soft state) means that an intermediate state, that is, a soft state, is allowed during a business operation, which does not require atomicity, either to complete or not to complete. For example, when placing an order, there is a status of "in process". Because of this soft state, my consistency does not require strong consistency, but final consistency, that is, as long as the final request can be processed, all data states are processed; if something goes wrong during the period, all the data are consistent, the failure of the failure, the refund of the money, the reset of the reset.

Implementation of distributed transaction

Therefore, after determining that the implementation principle of the distributed system is the ultimate consistency, it is also clear that we realize the principle of distributed transactions, which is also the ultimate consistency.

In fact, whether it is the ACID characteristics of database transactions or the ultimate consistency of distributed transactions, they are all implemented in different ways according to the definition of the transaction and its two goals.

So how should we achieve this ultimate consistency?

Single service distributed transaction

First of all, any distributed system is always composed of systems, that is, services, which can be deployed. At the same time, our whole system also needs certain ways to interact and communicate with each other. Sometimes, we can have one service directly invoke the interface of another service, if provided, and sometimes we can have two services communicate through a messaging middleware such as MQ to do some business together. However, in most cases, a service in a distributed system always accesses multiple data sources. The most typical example is to accept an event through MQ, then take some action, and then send the result to another queue.

For this situation where each service accesses multiple data sources, it is actually the simplest distributed transaction scenario. If you search for "Spring distributed transaction implementation" on the Internet, the results are all about the distributed transaction implementation process in this scenario.

To implement this transaction, you first need to have some understanding of the transaction mechanism of Spring. In this case, the simplest thing is to use JTA transaction management with Spring. However, we know that JTA transaction management is implemented through two-phase commit, and in many cases it is inefficient. Because when multiple data sources modify data, the data is always in a locked state, and it will not be released until the transactions of multiple data sources have been committed.

If we don't use JTA,Spring, it also provides us with several ways to implement distributed transactions approximately (note the approximation here). For example:

Transaction synchronization, that is, when one thing is committed, another transaction is notified by Listener and so on that another transaction is also committed. But in this case, if something goes wrong when the second transaction commits, the first thing cannot be rolled back because it has already been committed.

Chained transaction, that is, multiple transactions are packaged in a chained transaction manager, and when the transaction is committed, the transaction is committed at once. For this kind of implementation, there are also problems mentioned above.

There are other ways not to explain too much.

Therefore, using Spring to implement distributed transactions in the case of single service and multiple data sources, there is no way to fully implement transactions, because there is no guarantee that they will roll when something goes wrong. At this time, it needs to be supplemented by other mechanisms.

The first is to retry, that is, if something goes wrong, retry the previous operation. This is more common when there is a MQ, because the general MQ server, after you read the message, if there is an error in the processing, then the read message operation will not be submitted. Then the message will be re-read and restart the operation just now. At this point, we need to consider the idempotency of this method to ensure that the data will not be processed repeatedly when the message is repeated.

Second, we need to deal with some mistakes ourselves. For example, in the above case, after several retries, it has not been successful, then you need to follow the failure logic at this time. Sometimes, we can also use a timer to check for failed operations that have not been completed within a certain period of time.

In some cases, we also need to consider a variety of other errors, such as network errors, timeouts, system outages, and so on.

You can imagine that the more complex a distributed system is, the more errors it has, and the more remedial measures we need to consider. Then this kind of tinkering to achieve the ultimate consistency of distributed transactions is never a good way. However, using Spring to solve the distributed system of single service is always the foundation of distributed transaction implementation. We can use other modes to facilitate us to solve distributed transactions, but in each service, we often use transaction synchronization, chained transactions, etc., to implement transactions. We use Spring to guarantee transaction problems in most cases, while for special error cases, we use other modes to solve them.

The pattern of distributed transaction implementation

As I just said, we use other patterns to feel the problem of distributed transactions, so what are the patterns?

Message driven (Event Driven) mode

The message-driven pattern is that when a business request needs to be completed by multiple services, these services do not communicate directly, but through a MQ middleware. For example, for a request for payment of an order, after receiving the request for completion of the payment, through MQ, notify the order service to complete the order, and then notify the commodity service to reduce inventory, and then notify the logistics service to initiate the logistics process.

Then, for each service, it is necessary to read a message from one queue, complete its own business operation, and then send a message to another queue, which requires operating a data and a MQ server. This is the distributed transaction implementation of the single service mentioned above. For this model, we use transaction synchronization to ensure that transactions are guaranteed in most cases in each service. Even if there are occasional network errors, system errors, etc., most of the problems can be solved by retrying. If the retry doesn't work out, then deal with the failure logic.

When we use this approach, the most important thing is to orchestrate the message and its processing flow, and secondly, it is also a responsive programming thinking.

Event traceability (Event Sourcing) mode

Based on the message-driven mentioned above, Event Sourcing further enhances the status of the event (that is, the previous message) and makes it a first-class citizen of the system. In other words, the system is not based on those entities, but based on events, and an event represents a change in the state of a business operation and business data. As for business data, we do not need to save it in the database, even if it is saved, it is only for the convenience of querying the data.

In the Event Sourcing pattern, each service completes a certain logic in much the same way as the message-driven pattern mentioned above, that is, for each operation of a user, an event (possibly multiple) is generated, which is handled by a processing method of a service, or it may generate other events and then be handled by other services until the entire business process is completed. However, the biggest difference between it and message-driven is that in Event Sourcing services, business state data does not have to be stored in the database, even if it is saved, it doesn't matter if it goes wrong, anyway, it can be regenerated according to Event events. So for the affairs of this place, we just need to make sure that the Event is saved successfully. Of course, we need other mechanisms to enable us to regenerate business data, which is generally provided by a framework that implements Event Sourcing.

TCC (Try-Confirm-Cancel) mode

In addition to the above-mentioned association of different services through an intermediate price, in some distributed systems, our different services can communicate directly. For example, the Spring Cloud micro-service framework provides Rest access to other services. Then, at this time, it is equivalent to that one of my services not only accesses its own database, but also accesses other services. This service here can be regarded as a database. We may have to call an interface of another service to complete some business.

In this case, it is impossible for a service to provide an interface to implement a transaction, that is, to manipulate the data first, then Commit, and then Rollback if something goes wrong. However, we can learn from this idea of transaction processing to provide our own method of similar transactions, which is the TCC pattern. One thing is implemented through Do-Commit/Rollback. In the TCC pattern, it provides a set of Try-Confirm/Cancel interfaces to the operation interfaces invoked between each service.

Another example is that the payment is completed after the user places the order. When the payment is completed, the order service is processed first, and then the commodity service is called to reduce inventory. If you use Spring Cloud, you may write an interface in goods and services to directly reduce inventory, but in TCC mode, we need three interfaces. The first is the Try interface for inventory reduction. Here, we need to check the status of the business data, check whether the inventory of goods is sufficient, and then reserve resources, that is, set the reserved status on a certain field. Then in the Confirm interface, complete the operation of inventory minus 1. In the Cancel interface, reset the previously reserved fields.

This may sound a bit tedious and feel like something can be done at once, why should it be divided into two steps? first, it is to reset the inventory data correctly when something goes wrong. Secondly, the reservation operation and the Confirm operation are two requests, and there may be other concurrent requests in between. In theory, as long as the logic of reserving resources in the Try interface is correct, even if something goes wrong with Confirm, I can do it by retrying the Confirm request

Use a database to save transaction state

It's not really a pattern, it's just a way. For example, in TCC mode, when the Confirm interface is ready to be called, the target service suddenly goes down, or the service that initiates the request suddenly goes down or goes wrong, so that the Confirm request has not been called. So, after the system is restored, how can I complete the previous transaction? In addition to periodically checking unfinished operations with timers as mentioned above (we need to be able to judge by some data status that the business has not been completed), we can also use the database to record the running status of transactions.

For example, in TCC mode, whenever one service A wants to call another service B using TCC pattern, Service A writes the transaction state of the TCC to the database. Depending on the implementation, it may record the state of the current transaction before the call, and then save the parameters and result state of the call after the call is completed. After the transaction is completed (that is, after the call of Confirm or Cancel), it is updated to the completed state. Then, through reasonable design, we can guarantee that we can continue to complete the transaction or cancel the transaction in all kinds of error situations.

The above is how to implement distributed transactions in Spring. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.