Shulou(Shulou.com)06/02 Report--

This article shares with you about how event-driven microservices architecture can be used to achieve ultimate consistency. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Transaction consistency

First, let's review the ACID principles:

Atomicity: atomicity, changing the state of data is either done together or fails together

Consistency: consistency, the state of the data is completely consistent

Isolation: isolation line, which does not affect each other even if there are concurrent transactions

Durability: persistence. Once the transaction is committed, it cannot be undone.

In single applications, we can use the characteristics of relational database to achieve transaction consistency, but once the application develops to micro-services, it is divided into unused modules according to the business, and the database of each module has been separated, at this time, we have to deal with distributed transactions, and we need to complete ACID in the code. The more popular solutions are: two-phase commit, compensation mechanism, local message table (using local transactions and MQ), MQ transaction messages (RocketMQ).

CAP theorem

In 1998, Eric Brewer, a computer scientist at the University of California, proposed that there are three indicators for distributed systems.

Consistency: consistency

Availability: availability

Partition tolerance: partition fault tolerance

Eric Brewer says it is impossible to achieve all three indicators at the same time. This conclusion is called CAP theorem.

In micro-services, the databases used between different modules are different, and the services deployed between different modules may not be used, so partition fault tolerance is inevitable, because the invocation between services cannot be guaranteed to be 100% clean. So the system design must consider this situation. Therefore, we can assume that the P of CAP is always true, and the rest of C and A can't do it at the same time.

In fact, according to the CAP principle in distributed systems, when P (partition tolerance) occurs, the forced pursuit of C (consistency) will lead to the decline of (A) availability and throughput. At this time, we generally use the final consistency to ensure the AP capability of our system. Of course, it is not to give up C, but to give up strong consistency, and in general, CAP can guarantee, but in the case of partition fault tolerance, we can ensure data consistency through final consistency.

Event-driven to achieve ultimate consistency

The event-driven architecture synchronizes the state between domain objects through asynchronous messages, and some messages can be published to multiple services at the same time. After the message causes the synchronization of one service, it may cause another message, and the event will spread. Strictly speaking, event-driven is not called synchronously.

Example:

In e-commerce, users who place an order must determine whether the order is closed according to inventory.

Project architecture: SpringBoot2+Mybatis+tk-Mybatis+ActiveMQ [not made into Spring Cloud architecture because of small examples]

First, let's look at normal calls between services:

Code:

@ Override @ Transactional (rollbackFor = Exception.class) public Result placeOrder (OrderQuery query) {Result result = new Result (); / / call Stock-Service remotely to reduce inventory RestTemplate restTemplate = new RestTemplate (); / / request header HttpHeaders headers = new HttpHeaders (); headers.setContentType (MediaType.APPLICATION_JSON); / / encapsulated into a request object HttpEntity entity = new HttpEntity (query, headers) / / Interface Result stockResult = restTemplate.postForObject for calling inventory service synchronously ("http://127.0.0.1:8081/stock/reduceStock",entity,Result.class); if (stockResult.getCode () = = Result.ResultConstants.SUCCESS) {Order order = new Order (); BeanUtils.copyProperties (query,order); order.setOrderStatus (1); Integer insertCount = orderMapper.insertSelective (order); if (insertCount = = 1) {result.setMsg (" order placed successfully ") } else {result.setMsg ("failed to place order");}} else {result.setCode (Result.ResultConstants.FAIL); result.setMsg ("failed to place order:" + stockResult.getMsg ());} return result;}

We can see that there are many disadvantages to such a service invocation:

1. The order service needs to wait for the return result of the inventory service at the same time, and the API result returns delay. 2. The order service directly depends on the inventory service. As long as the inventory service collapses, the order service can no longer operate normally. 3. Concurrency should be considered in order service, and inventory may be negative in the end.

Let's start using event-driven to achieve final consistency.

1. After a new order is added to the order service, the status of the order is "opened", and then an Order Created event is issued to the message queue

Code:

@ Transactional (rollbackFor = Exception.class) public Result placeOrderByMQ (OrderQuery query) {Result result = new Result (); / / first create an order with a status of 0 Order order = new Order (); BeanUtils.copyProperties (query,order); order.setOrderStatus (0); Integer insertCount = orderMapper.insertSelective (order); if (insertCount = = 1) {/ / send order message MqOrderMsg mqOrderMsg = new MqOrderMsg (); mqOrderMsg.setId (order.getId ()); mqOrderMsg.setGoodCount (query.getGoodCount ()) MqOrderMsg.setGoodName (query.getGoodName ()); mqOrderMsg.setStockId (query.getStockId ()); jmsProducer.sendOrderCreatedMsg (mqOrderMsg); / / the order at this time is only open status result.setMsg ("order placed successfully");} return result;}

2. The inventory service listens to the message in the message queue OrderCreated, subtracts the order quantity from the inventory of the items in the inventory table, and then sends a Stock Locked event to the message queue.

Code:

/ * * receive the message of placing an order * @ param message received the message * @ param session context * / @ JmsListener (destination = ORDER_CREATE,containerFactory = "myListenerContainerFactory") @ Transactional (rollbackFor = Exception.class) public void receiveOrderCreatedMsg (Message message, Session session) {try {if (message instanceof ActiveMQObjectMessage) {MqStockMsg result = new MqStockMsg (); ActiveMQObjectMessage objectMessage= (ActiveMQObjectMessage) message; MqOrderMsg msg = (MqOrderMsg) objectMessage.getObject () Integer updateCount = stockMapper.updateNumByStockId (msg.getStockId (), msg.getGoodCount ()); if (updateCount > = 1) {result.setSuccess (true); result.setOrderId (msg.getId ());} else {result.setSuccess (false);} / / Manual ack to take messages out of queue, otherwise message.acknowledge () will be continuously consumed; / / send inventory lock messages to MQ jmsProducer.sendStockLockedMsg (result) }} catch (JMSException e) {log.error ("receiving order creation message error:" + e.getMessage ());}}

Careful friends may see: message.acknowledge (), that is, manually confirm the message. Because confirming that the message has been consumed after ensuring that the logic of the inventory service can be executed normally, we can ensure the delivery reliability of the message. In case there is an exception in the inventory service execution times, we can re-consume the order message.

3. The order service receives the Stock Locked event and changes the status of the order to "confirmed"

Code:

/ * determine whether there is stock, update order status is 1 with inventory, update order status with no inventory is 2, and notify user (WebSocket) * @ param message * / @ JmsListener (destination = STOCK_LOCKED,containerFactory = "myListenerContainerFactory") @ Transactional (rollbackFor = Exception.class) public void receiveStockLockedMsg (Message message, Session session) {try {if (message instanceof ActiveMQObjectMessage) {ActiveMQObjectMessage objectMessage= (ActiveMQObjectMessage) message; MqStockMsg msg = (MqStockMsg) objectMessage.getObject () If (msg.isSuccess ()) {Order updateOrder = new Order (); updateOrder.setId (msg.getOrderId ()); updateOrder.setOrderStatus (1); orderMapper.updateByPrimaryKeySelective (updateOrder); log.info ("order [" + msg.getOrderId () + "] order placed successfully");} else {Order updateOrder = new Order (); updateOrder.setId (msg.getOrderId ()); updateOrder.setOrderStatus (2); orderMapper.updateByPrimaryKeySelective (updateOrder) / / notify the user that the inventory is insufficient and the order is cancelled log.error ("order [" + msg.getOrderId () + "] is cancelled due to insufficient inventory");} / / manually ack to make the message dequeue, otherwise message.acknowledge () will be consumed continuously;}} catch (JMSException e) {log.error ("receiving inventory lock message error:" + e.getMessage ());}}

Similarly, here we also use manual confirmation messages to ensure the reliability of message delivery.

At this point, it's all done. Let's take a look at how it compares to a normal service call:

1. The order service no longer directly depends on the inventory service, but sends the order event to the MQ for inventory monitoring.

2. The order service can really run independently as a module.

3. The concurrency problem is solved, and the queue processing efficiency of MQ is very high.

But there are also the following problems:

1. The user experience has changed: because of the event mechanism, the order is generated immediately, but it is likely that after a while, the system will remind you that you are out of stock. This is like waiting in line to rush to buy, waiting in line and being told that it is out of stock, so you don't have to wait in line anymore.

2. There may be a lot of orders in the database that have not been completed.

Finally, what if you really want to consider the user experience and don't want a lot of unnecessary data in the database?

Then put the order service and inventory service together. Solving the current problem should be the first consideration. We designed the microservice to solve the business concurrency. Now we are faced with the problem of user experience, so the architecture design also needs to be compromised.

The most important thing is that we have thought and analyzed to what extent each scheme can be achieved and which scenarios can be applied. As the saying goes, technology should be combined with the actual scene, and we can't copy it mechanically in order to pursue new technology.

Thank you for reading! This is the end of the article on "how to use event-driven to achieve ultimate consistency in micro-service architecture". I hope the above content can be of some help to you, so that you can learn more knowledge. If you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.