Shulou(Shulou.com)06/01 Report--

Micro-service architecture is becoming more and more popular. It is a method of modularization. It splits a whole application into services. It allows teams to deliver high-quality software faster when developing large and complex applications. Team members can easily accept new technologies because they can use the latest and recommended technology stacks to implement their services. The micro-service architecture also improves the scalability of applications by allowing each service to be deployed on the best-state hardware.

But microservices are not everything. Especially in domain models, transactions, and queries, it seems that splitting is not always suitable. In other words, these are also areas where micro-services need to be specially dealt with, compared to the single architecture in the past.

In this article, I will describe a way to develop micro-services that can solve these problems. It is mainly designed through domain models, that is, DDD and event sources (Event Sourcing) and CQRS. Let's first take a look at what problems developers will encounter when developing microservices.

Challenges in the process of microservice development

Modularization is very necessary when developing large and complex applications.

Nowadays many applications are too big to be done by one person. And it's so complicated that it's impossible to understand it by one person.

In this case, the application must be split into modules. In monolithic applications, a module is defined as, for example, a java package. However, this practice is not ideal in practice, and the monomer application becomes larger and larger over a long period of time. The microservice architecture treats the service as a modular unit.

Each service corresponds to a business capability that the organization needs to create value. For example, micro-service-based online stores include a variety of services, including ordering service (Order Service), customer service (Customer Service), and directory service (Catalog Service).

Every service has a boundary that cannot be violated and is difficult to violate. That is, each microservice should provide a separate and independent capability. In this way, the modularization of the application is easier to save over time.

The microservice architecture has other advantages. These capabilities include deploying services independently, expanding services independently, and so on. Compared to monomers.

Unfortunately, splitting is not as easy as it sounds. It's pretty hard.

The applied domain model, transaction, and query are the splitting problems you face during and after the split. Let's take a look at the specific reasons.

Problem 1-split Domain Model

Domain model pattern is a very good way to implement complex business logic. For example, for an online store, the domain model will contain several classes: Order, OrderLineItem, Customer and Product. In the micro-service architecture, the Order and OrderLineItem classes are part of Order Service; Customer is part of Customer Service; and Product is part of Catalog Service.

One of the challenges of splitting domain models is that class usually references one or more other classes.

For example, the order class references the customer Customer;OrderLineItem of the order, which refers to the product Product ordered by the order.

What should we do with these references that we want to cross service boundaries?

Later you will see a concept from domain model design: Aggregate. We solve this problem through aggregation.

Microservices and databases

A very obvious feature of the microservice architecture is that the data owned by a service can only be accessed through the service's API.

In an e-commerce site, for example, OrderService owns a database with a table ORDERS;CustomerService and its own database containing the table CUSTOMERS.

With this encapsulation, microservices are decoupled.

During development, developers can independently modify the database shema of their own services without the need for coordination and blending with the development of other services.

In production, services are isolated. For example, a service never blocks waiting because another service holds a lock on the database.

Unfortunately, this database split makes it difficult to manage data consistency and cross-table queries between different services.

Problem 2-Cross-service distributed transaction implementation

A traditional monolithic application can enforce business rules through ACID transactions to achieve consistency.

Imagine, for example, that all users in e-commerce have a credit line, that is, they have to look at the credit before creating an order.

The application must ensure that potential multiple concurrent attempts to create orders do not exceed the customer's credit limit.

If Orders and Customers are both in the same library, you can use ACID transactions to do this:

BEGIN TRANSACTION... SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID =? SELECT CREDIT_LIMITFROM CUSTOMERS WHERE CUSTOMER_ID =? INSERT INTO ORDERS... COMMIT TRANSACTION

Unfortunately, we cannot manage data consistency in this way in a micro-service architecture.

The ORDERS and CUSTOMERS tables are owned by different services and can only be accessed through their respective service API. They may even be in different databases.

One of the more common practices is to use distributed transactions, such as 2PC, etc. However, this approach may not be a feasible solution for modern applications. CAP Theorem requires you to choose between usability and consistency, and usability is usually a better choice.

Moreover, many modern technologies, such as most NoSQL databases, do not even support ACID transactions, let alone 2PC.

So managing data consistency needs to be done in other ways.

You will see later that we use a technique in the event-driven architecture called event sources (event sourcing) to solve distributed transactions.

Question 3-query

Managing data consistency is not the only challenge. Another problem is the query problem.

In traditional monomer applications, we usually use join to implement cross-table queries.

For example, we can easily query the recent large orders placed by our customers through the following sql:

SELECT * FROM CUSTOMER c, ORDER oWHERE c.id = o.ID AND o.ORDER_TOTAL > 100000 AND o.STATE = 'SHIPPED' AND c.CREATION_DATE >?

However, we cannot implement such a query in a micro-service architecture.

As mentioned earlier, the ORDERS and CUSTOMERS tables belong to different services and can only be accessed through the service API.

And they may be using different databases.

Moreover, even if you use the event source (Event Sourcing) to handle query problems, it may be more troublesome.

Later, you will learn that one solution is to solve distributed query problems through a practice called CQRS (Command Query Responsibility Segregation).

But first, let's take a look at the domain driven design (DDD) tool, where it is necessary to develop business logic based on the domain model under our microservice architecture.

DDD aggregation is the building block of microservices

As you can see, in order to successfully develop business applications using the micro-service architecture, we have to solve the problems mentioned above.

The solutions to these problems can be found in Eric Evans's book Domain-Driven Design.

This book, published in 2003, mainly introduces some methods of designing complex software. These methods are also useful for developing micro-services.

In particular, domain-driven design allows you to create a modular domain model that can be used by multiple microservices.

What is aggregation?

In domain-driven design, Evans defines several building blocks for the domain model.

Many have become part of the everyday developer language, including entity, which refers to a persistent object with a unique identity. Value object, that is, VO, as you often hear, is used to store data, either corresponding to database tables or not, similar to DTO used to transfer data. Service refers to a service that contains business logic. But it should not be classified as entity or value object.

Repository, which means that a collection of entity is a repository.

Building block and aggregate are often ignored by developers, except for DDD enthusiasts, or "fanatics".

However, aggregate has proved to be the key to the development of microservices and is very important.

An aggregate is a collection of domain that can be treated as a unit. One of the units here is that it can be treated as an atom.

It contains a root entity and possibly one or more associated entity and value object.

For example, there are several aggregations for domain model for an online store, such as Order and Customer.

Order aggregation consists of a root entity Order and more than one OrderLineItem value object, and OrderLineItem may also be associated with other vo, such as express address (Address) and payment account information PaymentInformation.

Customer aggregation consists of a root entity Customer and other vo such as DeliveryInfo and PaymentInformation.

Using aggregation to disperse and participate in the domain model (domain model) in each aggregation also makes the domain model easier to understand. This also clarifies the scope of the operation, such as query operation and delete operation.

An aggregation is usually load out of the database as a whole. Delete an aggregation, that is, delete all the object in it.

However, the benefits of aggregation go far beyond modularization of a domain model. This is because aggregation must follow certain rules.

References between aggregates must use a primary key

The first rule is that aggregates are referenced by id (such as primary keys) rather than by object references.

For example, Order references Customer through customerId, not Customer's object.

Similarly, OrderLineItem references Product through productId.

This approach is very different from traditional object modeling. Although the latter thinks it seems strange to do so in the domain model through foreign key references.

By using ID instead of object references, it means that the aggregation is loosely coupled. You can easily put different aggregations on different service.

In fact, the business logic of a microservice consists of a domain model. This domain model is a combination of several aggregations. For example, OrderService includes Customer aggregation.

A transaction creates or updates only one aggregation

The second rule is that aggregations must follow that one transaction can only create or update one aggregation.

When I first looked at these rules, I didn't feel anything at that time. Because at that time, I was still developing traditional monolithic applications, RDBMS-based applications. So the transaction can update any data. Today, these constraints still apply to micro-service architectures. It ensures that a transaction is contained only in a microservice. This constraint also conforms to the limited transaction model of most NoSQL databases.

When developing a domain model, it is important that you determine how big each aggregation is.

On the one hand, aggregation should ideally be small. It improves modularity by separating concerns.

This is more efficient because aggregations are usually fully loaded.

In addition, because updates to each aggregation occur sequentially, the use of fine-grained aggregations increases the number of concurrent requests that the application can handle, thereby improving scalability.

It will also improve the user experience because it reduces the likelihood that two users will try to update the same aggregation.

On the other hand, because aggregation is the scope of the transaction, you may need to define a larger aggregation to atomize specific updates.

For example, I described earlier that in the online store domain model, Order and Customer are independent aggregates.

Another design could be to use Orders as part of Customer aggregation.

The benefit of a larger Customer aggregation is that the application can force atomic verification of credit lines. The disadvantage of this approach is that it combines order and customer management functions into the same service. This also reduces scalability because transactions that update different orders for the same customer will be sequenced.

Similarly, it is possible for two users to try to edit different orders placed by the same customer. Also, as the number of orders increases, the cost of loading an Customer aggregation becomes more expensive.

Because of these problems, it is best to make the aggregation as fine-grained as possible.

Even if a transaction can only create and update a single aggregation, the consistency between aggregates must still be managed in microservice applications.

It must be verified in the Order service that a new Order aggregation will not exceed the credit line of the Customer aggregation.

There are two different ways to address consistency.

One approach is to spoof the creation and / or update of multiple aggregations in a single transaction. This approach is based on the premise that all aggregations are owned by one service and that these aggregations are persisted in the same RDBMS.

Another approach is to use an ultimately consistent event-driven (event-driven) approach to maintain consistency between aggregates.

Using event drivers to maintain data consistency

In modern applications, there are various constraints on transactions, which makes it difficult to maintain data consistency between services.

Each service has its own private data, so 2PC's solution becomes unfeasible.

More importantly, many applications use NoSQL databases, which do not support local ACID transactions at all, let alone distributed transactions.

Therefore, modern applications must use an event-driven and ultimately consistent transaction model.

What is an Event?

According to Merriam-Webster, the meaning of an event is: something that happens:

In this article, we define domain events as events that aggregate. An event (event) usually indicates a change of state. Now let's take e-commerce system as an example, an Order aggregation. Its status change events include order created (Order Created), order cancelled (Order Cancelled), and order placed (Order Shipped). An event can indicate an action that violates business rules, such as the credit line of a customer (Customer).

Use the Event-Driven schema

Services use events to manage consistency between aggregates, such as a scenario like this: an aggregation publishing event, such as a state change of the aggregation or an attempt to violate business rules, and so on.

Other aggregators subscribe to this event and are then responsible for updating their own status.

The online store system verifies the customer's (customer) credit line when creating an order (order) using the following series of steps:

1. An order (Order) aggregation is created with a status of NEW, and an OrderCreated event is issued.

two。 The customer (Customer) consumes the OrderCreated event, then saves the credit value for the order, and then publishes a CreditReserved event.

3. The order (Order) aggregates the consumption CreditReserved event, and then modifies its status to APPROVED.

If the credit check fails due to insufficient funds, the customer (Customer) aggregates and publishes the CreditLimitExceeded event.

This event does not correspond to a change in state, but rather represents a failed attempt to violate business rules. Order (Order) aggregate consumption after this event, and change your status to CANCELLED.

Micro-service architecture can be compared to Web of event-driven aggregation

Under this architecture, the business logic of each service consists of one or more aggregations.

A transaction can contain only one service, and is to update or create a separate aggregation. That is, the transaction within the aggregation.

Services manage consistency between aggregates by using events.

One of the obvious benefits of this approach is that aggregations turn into loose and decoupled building blocks.

They can be deployed as a single application or as a set of services.

In this case, you can use a monolithic architecture at the beginning of a project.

After that, as the size of the application and the size of the development team grow, you can easily switch to the micro-service architecture.

Summary

Microservice architecture divides a whole application into services functionally, and each service corresponds to a business capability. When we develop business applications based on micro-service architecture, one of the key challenges is transactions, domain models and queries, all of which are the problems caused by the split. You can split the domain model by using the concept of DDD aggregation. The business logic of each service is a domain model, which then consists of one or more DDD aggregations.

In each service, a transaction can only create or update a separate aggregation. Since 2PC is not a viable solution for modern applications, we need to use event mechanisms to achieve consistency between aggregates (and between services). In the next episode, we will describe the use of event sourcing to implement an event-driven architecture. We will also show you how to use CQRS to implement queries in a micro-service architecture.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.