Shulou(Shulou.com)06/01 Report--

This article is about sample analysis of RocketMQ clustering processes and core concepts. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

1. Why there is a single point of failure in the cluster? the cluster can share the pressure. Improving the master and slave of QPS can ensure the reliability of messages, for example, only M but no S. M disk is broken, and the unconsumed messages are lost. And S can be used as a backup. Second, single M mode 1, characterized by only one Master node, so a single point of failure is a fatal disadvantage. Advantages: simple configuration and convenient deployment. Disadvantages: a single point of failure, once the Broker restarts or goes down directly, it will cause the entire service to be unavailable. 2. Illustration

3. Multi-M mode 1. A cluster has no Slave nodes, all are Master nodes. Advantages: the configuration is relatively not complex, a single M downtime or restart is not aware of the business system, and provide services as usual. It's just that if a message is not consumed on this broker, it may not be able to continue to consume, but the message will not be lost and persisted to disk. If you swipe the disk asynchronously, there will be a small amount of loss. Disadvantages: during the downtime of a single machine, messages that are not consumed on this machine cannot be subscribed until the machine is restored, and the real-time performance of messages will be affected. 2. Illustration

Fourth, multi-M and multi-S mode

A cluster has both Master nodes and Slave nodes.

1. Asynchronous replication each Master is configured with one Slave, with multiple pairs of Master-Slave and HA. Asynchronous / synchronous replication is used. The master / slave has a short message delay of millisecond. Advantages: even if the disk is damaged, very few messages are lost, and the real-time performance of messages will not be affected, because after Master downtime, consumers can still consume from Slave, this process is transparent to the application. No human intervention is required. The performance is almost the same as the multi-Master mode. Disadvantages: Master downtime, disk damage, will lose a small number of messages. 2. Synchronous double writing

Each Master is configured with a Slave, and there are multiple pairs of Master-Slave. HA uses synchronous double-write mode. Both master and slave are successfully written, and success is returned to the application.

Advantages: there is no single point for both data and service. In the case of Master downtime, there is no message delay, and the service availability and data availability are very high.

Cons: performance is slightly lower than asynchronous replication mode, about 10% lower, and the RT for sending a single message is slightly higher. At present, after the master downtime, the slave cannot be automatically switched to the host, and the automatic switching feature will be supported later.

This synchronous double write is necessary to ensure that the message is not lost.

3. Illustration

5. Distribution of queue

The queue of one topic can be distributed across multiple Broker. For example, if a topic has four queue, it may assign one queue to three queue,broker-b on the broker-a, and the allocation of this queue is determined by the broker side. But in order to verify the conjecture, we can manually create the topic from the console, and if successful, we can verify our conjecture.

First of all, I have a 2M2S cluster.

Create topic

Create topic

Looking at the status, we can see that we have created four queue on each broker, that is, a total of eight queue.

Click [TOPIC CONFIG] to change the configuration

If you look at it again, you will find that it has already taken effect and verified our conjecture.

Every queue message is different, that is, if you send N messages, he may be partly on broker-an and partly on broker-b. No matter where he is, the message is different. Don't interpret it as a copy of MMUS. It's just that the load balancer allocates queue to different broker.

Thank you for reading! This is the end of this article on "sample analysis of RocketMQ cluster processes and core concepts". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.