Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how to look at the message middleware from the business scenario". In the daily operation, I believe that many people have doubts about how to see the message middleware from the business scenario. The editor consulted all kinds of materials and sorted out the simple and useful operation methods. I hope it will be helpful for you to answer the doubt of "how to see message middleware from business scenarios"! Next, please follow the editor to study!

The three functions of MQ

The three major functions of MQ: business decoupling, asynchronous invocation, and traffic peaking.

Before we introduce messaging middleware, let's think about what is the most common method of calling between businesses without MQ. The more common methods are RPC and REST. A simple comparison of the two is as follows:

Because REST has high flexibility and low performance, while RPC has high performance and low flexibility, we can use REST for external interface and RPC for internal communication.

The architecture that uses RPC internally is as follows:

If it is a core application, there is no problem with the above architecture. However, if the business logic is added to the timely operation activity logic, it is obviously unreliable to modify the code logic frequently.

With the help of MQ, we take out the business logic of running the activity and put it behind MQ. All the requests initiated by the client are sent to MQ, and the business logic behind them is consumed according to their own needs. In this way, through MQ, we achieve business decoupling (operational activities and core business).

We mentioned above that many times the internal business call is RPC. In some scenarios, we can replace RPC with MQ to implement business asynchronous invocation.

So, can MQ completely replace RPC? In theory, it can replace RPC. But in this case, the stability of the whole system depends on MQ, there is a certain risk. While PRC is an independent framework, there is a problem with the business node, which has no impact on the overall situation. So it is not recommended to replace all RPC with MQ.

Next, let's look at a specific scenario (e-commerce scenario), how to implement business asynchronous calls through MQ.

In the following figure, before using MQ, the transaction service pushes services to sellers, records transaction data, and so on, through RPC. In fact, this has nothing to do with the core business logic. So it can be transformed into the mode of the second picture.

After introducing MQ's implementation of business logic split and asynchronous invocation, let's take a look at the scenario where MQ implements traffic peaking.

JD.com often holds second-kill activities. At the beginning of the second kill, many users made requests and the number of visits soared. At this time, in order to ensure the smooth operation of the business logic layer, it is necessary to add a MQ between the API gateway and the salesman logic. Client requests are piled on the MQ and then processed by the business logic. At this point, we can open up an API for querying the results in seconds to the user, and call the business logic layer through RPC.

Although MQ can achieve traffic peaking, traffic decoupling and asynchronous invocation, the introduction of MQ also has all the disadvantages, such as increasing the complexity of fault diagnosis and architecture. Therefore, MQ can not completely replace RPC, the two should be used together. In the micro-service scenario, the MQ is usually placed between the API gateway and the business logic layer, as shown in the figure above.

Comparison of several kinds of MQ

Next, let's compare the advantages and disadvantages of several mainstream MQ:

From the figure above, we can see that RocketMQ has certain advantages in terms of functionality. Rocket is currently a popular MQ open source project:

If you want to compare the application scenarios of Kafka and RocketMQ, to put it simply: Kafka does not do business things, it does not guarantee business consistency, it is a data flow channel between systems; RocketMQ does business-related things, it ensures business consistency, and it implements high-performance and reliable information transmission.

Normal messages are sent by load balancing. A Topic spans multiple queues. If you want sequential messages, you must send them to a queue, and only one thread consumes the messages.

Several core reasons why RocketMQ is widely used are: support for transaction messages, support for delayed messages, support for message retransmission, support for client-side tag filtering, and support for message playback.

Topology and Logic Diagram of RocketMQ

The topology diagram of RocketMQ is as follows.

Each node in the Name Server cluster of RocketMQ is independent of each other and is responsible for the service registration discovery of the MQ cluster.

The logical diagram is as follows:

Because RocketMQ cannot achieve master-slave switching, message producers are only connected to the master and consumers are connected to the master and slave.

Combined with the reliability and availability of RockerMQ, our usual use is shown in the following figure, that is, multi-Master and multi-Salve+ asynchronous replication.

In RockerMQ, message producers write messages in commitlog by appending them sequentially. Commitlog is the body of the message. The dispatch thread then reads the offset and size of the message from the commitlog and puts it into a consumption queue in the form of a queue by topic. That is, the queue does not put the message body. Instead, it is the index information of the message.

The CommitLog is stored as a physical file, and the CommitLog on each Broker is shared by all ConsumeQueue on the machine. In order to improve the performance of message reading, RocketMQ splits the commitlog at 1G by default. In addition, for MQ systems, try to use SDD or NVRAM.

The message delivery mode of RocketMQ

We know that there are two main modes of message delivery between services: Queue and Topic.

Queue mode is an one-to-one transmission mode. In this mode, the producer of the message (Producer) delivers the destination type of the message as Queue. In Queue, a message can only be delivered to one consumer (Consumer). If no consumer is listening to the queue, the message will remain in the queue until the message consumer connects to the queue, and the consumer will request a message from the queue-> consumer pull mode.

Topic mode is an one-to-many message transmission mode. In this mode, the producer of the message (Producer) delivers the destination type of the message as Topic. After the message arrives at Topic, the message server sends the message to all consumers who subscribe to this topic. -> push to consumer mode.

In RocketMQ, Topic is an abstract concept. There are N queue at the bottom of each Topic, and the data actually exists on the queue. Queue is one of the shares of Topic on a Broker after it is divided into specified portions. It is the basic unit of resource allocation in the process of load balancing. That is, a topic spans multiple queue, with each queue on one broker, so topic can span multiple broker.

In fact, both push and pull have their own advantages and disadvantages.

Push mode: the real-time performance of messages is high, but the consumption capacity of the client, that is, the speed of message processing is not taken into account.

Pull mode: the real-time performance of the message is low, and a large number of invalid requests may be made.

In order to learn from each other, RocketMQ adopts LongPoll mode, that is, long polling mode. In this way, the consumer initiatively sends a pull message, then broker hold the request and returns until there is a message. The timeout for the request is 30s. When the request times out, the consumer initiates the request again.

Consumer consumption style of RocketMQ

There are two main ways for consumers to consume news:

Cluster consumption: a single message is consumed only once, and Topic messages are evenly consumed in each stage.

Broadcast consumption: all messages are consumed in each cluster, and a single message is consumed once in each cluster.

At this point, the study of "how to look at the message middleware from the business scenario" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.