Shulou(Shulou.com)06/01 Report--

This article mainly shows you "what are the models of message middleware in GO language", the content is simple and easy to understand, and the organization is clear. I hope it can help you solve your doubts. Let Xiaobian lead you to study and learn "what are the models of message middleware in GO language" this article.

Message Oriented Middleware (MOM) is becoming increasingly important in enterprise development. This article introduces four message delivery models in message middleware, mainly introducing the core characteristics of the models and the differences between different models. These four models are:

PTP model

Pub/Sub model

Partition model

Transfer model

The PTP model and Pub/Sub model are defined in the JMS specification, and the messaging middleware ActiveMQ implements the JMS specification. However, some message middleware does not implement JMS specification, but designs a set of models by itself, for example, Kafka and RocketMQ adopt Partition model. There are also other messaging models in the industry, such as the Transfer model, which I named myself.

1. PTP model

Point-to-Point communication model. PTP is queue based, and a queue can have multiple producers and multiple consumers. Message servers queue messages in the order they are received. Each message in the queue can only be consumed by one consumer and removed from the queue after consumption.

It should be noted that although the concept of Queue is used here, it is not the message that enters the queue first, it must be consumed first. In the case of multiple downstream consumers, some message middleware, such as ActiveMQ, will distribute the messages in the queue to different consumers for parallel processing in order to improve the consumption capacity. This means that messages may be ordered when sent, but become disordered when consumed. To ensure orderly consumption, some MQs provide the concept of "exclusive consumers" or "exclusive consumers," in which case messages in the queue allow only one consumer to consume, and if there are multiple consumers, select one. However, this means that there is no parallelism in the processing of messages. If the message volume is large, a message backlog will occur. To solve the performance problem of "proprietary consumers," some message middleware uses the concept of partitioning to solve the performance problem, which we will describe later.

Pub/Sub model

publish-and- subscribe, i.e. the publish-subscribe model. In the Pub/Sub model, a producer publishes a message to a topic, and all downstream consumers subscribed to the Topic can receive the message. As shown below:

Usually, a message is consumed only once, so under what circumstances do all consumers need to consume this message? Typically, data needs to be cached in memory and updated in real time. For example, the author has done a forbidden word system to detect forbidden words in comments entered by users. This forbidden word system is deployed on N servers. In order to improve the detection performance, each machine will load the full amount of forbidden thesaurus into memory. The update of thesaurus is completed by sending MQ messages. Due to the Pub/Sub model, the consumer of each machine can receive this message and update the sensitive thesaurus directly in memory.

3. Partition model

To solve the performance problem that ordered messages need to be consumed by "proprietary consumers" under the PTP model, some messaging middleware, such as rocketmq and kafka, adopt the Partition model, i.e., the partition model, as follows:

When a producer sends a message to a Topic, it finally selects one of the partitions to send. You can think of partitions in the Partition model as queues in the PTP model, except that queues in the PTP model store all messages, and each Partition stores only part of the data. For the message, there is an additional concept of a consumer group. Partitions will be distributed among consumers in the same consumer group, and each consumer will only consume the Partitions assigned to them. The diagram above demonstrates that different consumers may be assigned different numbers of Partitions. Paritition mode cleverly combines PTP model and Pub/Sub model:

For PTP models:

A message is consumed by only one consumer, and each partition in the Partition model ends up with only one consumer. For scenarios where global consumption is ensured by "exclusive consumers," in the Partition model, only one Partition is needed to ensure that the Topic created, and this Partition will eventually allocate only one of the consumers. In addition, in most scenarios, we do not need to ensure global order, for example, an order generates three messages, namely order creation, order payment, and order completion. When consuming, it makes sense to consume in this order. However, orders can be consumed in parallel. For example, three messages generated by order 1 are sent to Partition 1, and three messages generated by order 2 are sent to Partition 2. In this way, parallel consumption between different orders is achieved.

For Pub/Sub models:

A message that all downstream consumers can consume. In the Partition model, you only need to set up a different consumer group for each consumer. However, too many consumer groups can cause problems for messaging middleware operations. So some message middleware combines the Partition model with the Pub/Sub model. For example, RocketMQ supports setting consumption mode for consumer groups. If it is cluster mode, consume according to the above description. If it is broadcast mode, consume according to Pub/Sub model. Of course, the Partition model is not all good, its biggest limitation is that the number of partitions is fixed (although it can be adjusted), and can only be assigned to one of the consumers. When the number of consumers is greater than the number of partitions, these extra consumers will not be able to consume the message. Some messaging middleware optimize for this, such as rocketmq, which supports parallel consumption of individual partitions. That is to say, within a single consumer, multiple threads are started at the same time to consume the data in this Partition. Of course, the premise is that the message is not ordered. For ordered messages, only one thread can consume the data in this Partition in sequence.

4. Transfer model

The concept of consumer groups in the Partition model is very useful. Messages under the same Topic can be consumed by multiple different business parties. As long as different consumer groups are used, the consumption positions of different consumer groups are recorded separately and do not affect each other. However, the Partition model limits the number of consumers to no more than the number of partitions. Therefore, there is another consumption model, which I call the Transfer model, as shown in the following figure:

Producers still send messages to Topics, and for a Topic, multiple channels can be created, called channels here. Unlike partitioning, every message sent to a Topic is forwarded to each channel, so each channel has the full amount of data for that Topic. Of course, there is no need to really copy a complete copy of the message body to the channel, you can only record the message metadata, indicating that there is a piece placed in this channel. When consumers consume messages, they must specify which channel to consume from. When multiple consumers consume the same channel, only one consumer consumes each message, similar to the PTP model. In fact, we can think of consumers who consume the same channel as automatically forming a consumer group. However, unlike the Partition model, there is no concept of partition, so the number of consumers can be arbitrary. In fact, NSQ message middleware written in GO language adopts this model. Of course, this model, like PTP, does not guarantee ordering of messages except through concepts similar to "dedicated consumers."

The above is all the content of this article,"What are the models of message middleware in GO language?" Thank you for reading! I believe that everyone has a certain understanding, hope to share the content to help everyone, if you still want to learn more knowledge, welcome to pay attention to the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.