Shulou(Shulou.com)06/03 Report--

Source: Redislabs

Author: Martin Forstner

Translation: Kev × ×)

In my experience, it is easier to build and maintain separate logical business services that are smaller, loosely coupled, and work together. These services (also known as microservices) manage their own technology stacks, so it is easy to develop and deploy independently of other services. Predecessors have summarized a lot about the benefits of using this architecture, so I'm not going to repeat it here. With regard to this kind of design, there is one aspect that I have been working on without it, which will lead to some interesting challenges. Although building loosely coupled microservices is a very lightweight and fast development process, the communication model for sharing state, events, and data among these services is not that simple. The simplest communication model I have ever used is direct communication between services, but this model has been clearly proved by Fernando Dogio that once the service scale expands, it will fail, resulting in service crashes, overloaded logic, and increased load, which may cause great trouble, so we should try to avoid using this model. There are other communication models, such as the general publish / subscribe model, the complex kafka event flow model, etc., but recently I have been using Redis to build a communication model between microservices.

Redis the savior!

The microservice publishes the state through the network boundary, and in order to track this state, events usually need to be stored in the event store. Because an event is usually a record of an immutable stream of an asynchronous write operation (also known as a transaction log), it applies to the following scenarios:

1. Order is important (time series data)

two。 Losing an event results in an error state

3. The playback state is known at any given point in time

4. Write operation is simple and fast

5. Read operations take more time, so much so that caching is needed

6. High scalability is required. Service is decoupled and there is no correlation.

With Redis, I can always easily implement the publish-subscribe model. But now that Redis 5.0provides a new Streams data type, we can model log data structures in a more abstract way-making it an ideal use case for time series data (such as transaction logs that pass semantics at most or at least once). Based on dual-host functions, easy deployment and ultra-fast processing capabilities in memory, Redis streaming has become a necessary tool for managing large-scale micro-service communications. The basic model is called Command query Separation of duties (CQRS), which executes commands and queries separately, using HTTP protocol for commands and RESP (Redis Serialization Protocol) for queries. Let's use an example to illustrate how to use Redis as an event store.

Overview of simple Application of OrderShop

I created a simple but general-purpose e-commerce application as an example. When creating / deleting customers, inventory items, or orders, use RESP to asynchronously pass events to the CRM service to manage OrderShop's interaction with current and potential customers. Like the requirements of many common applications, CRM services can be started and stopped at run time without affecting other micro services. This requires capturing all messages sent to it during its downtime for subsequent processing.

The following figure shows the interconnectivity of nine decoupled microservices that communicate between services using event storage built by Redis streams. They do this by listening for any newly created events on a specific event stream in the event store (that is, the Redis instance).

Figure 1. OrderShop architecture

The domain model of our OrderShop application consists of the following five entities:

customer

Product

Inventory

Order

Bill

By listening for domain events and keeping the entity cache up-to-date, the aggregation function of the event store needs to be called only once or in response.

Figure 2. OrderShop domain model

Install and run OrderShop

Follow these steps to install and run the OrderShop application:

1. Download the code repository here:

Https://github.com/Redislabs-Solution-Architects/ordershop

two。 Make sure Docker Engine and Docker Compose are installed

3. Install Python3:

Https://python-docs.readthedocs.io/en/latest/starting/install3/osx.html

4. Start the application using docker-compose up

5. Installation requirements using pip3 install-r client / requirements.txt

6. Then use python3-m unittest client / client.py to execute the client

7. Use docker-compose stop crm-service to stop the CRM service

8. Re-execute the client, and you will see that the application is running normally without any errors

In-depth understanding

Here are some simple test cases from client.py, along with the corresponding Redis data types and keys.

I chose the stream data type to hold these events because the abstract data type behind them is the transaction log, which is well suited to our use case of continuous event flow. I chose different keys to allocate partitions and decided to generate my own entry ID,ID for each stream with a "-" microsecond timestamp (to keep the ID unique and keep the order of events between keys / partitions). I choose collections to store ID (UUID) and lists and hashes to model the data because it reflects their structure and the entity cache is just a simple projection of the domain model.

Conclusion

The various data structures provided by Redis-including collections, ordered collections, hashes, lists, strings, bit arrays, HyperLogLogs, geospatial indexes, and now streams-can easily adapt to any data model. The stream contains elements that are not only a single string, but also objects consisting of fields and values. The range query is fast, and each entry in the flow has an ID, which is a logical offset. Streaming provides solutions for applications such as time series, and can provide streaming messages for other applications, such as replacing generic publish / subscribe applications that require higher reliability, as well as other entirely new applications.

You can expand Redis instances and provide persistence options for disaster recovery by sharding (aggregating multiple instances), so Redis can be used as a choice for enterprise applications.

More high-quality middleware technical information / original / translated articles / materials / practical information, × ×

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.