Shulou(Shulou.com)06/02 Report--

This article will explain in detail what is the principle of RocketMQ master-slave read-write separation mechanism. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

RocketMQ's read-write separation mechanism is not consistent with the above description. RocketMQ has its own set of read-write separation logic, which determines the amount of messages accumulated on the main server to determine whether consumers consume messages from the server.

The value that determines whether the consumer consumes messages pulled from the server is stored in the GetMessageResult class:

Org.apache.rocketmq.store.GetMessageResult:

Private boolean suggestPullingFromSlave = false

The default value is false, which means that the default consumer will not consume the slave server. The following logic can change this value:

Org.apache.rocketmq.store.DefaultMessageStore#getMessage:

Long diff = maxOffsetPy-maxPhyOffsetPulling;long memory = (long) (StoreUtil.TOTAL_PHYSICAL_MEMORY_SIZE * (this.messageStoreConfig.getAccessMessageInMemoryMaxRatio () / 100.0)); getResult.setSuggestPullingFromSlave (diff > memory)

MaxOffsetPy is the current maximum physical offset, and maxPhyOffsetPulling pulls the maximum physical offset for this message, and their difference represents the message accumulation. TOTAL_PHYSICAL_MEMORY_SIZE represents the current system physical memory, and the default value of accessMessageInMemoryMaxRatio is 40. The above logic can calculate whether the current message accumulation is greater than 40% of the physical memory. If it is larger, set suggestPullingFromSlave to true.

Next, the parameter value plays a role in the message pull logic:

Org.apache.rocketmq.broker.processor.PullMessageProcessor#processRequest:

If (getMessageResult.isSuggestPullingFromSlave ()) {responseHeader.setSuggestWhichBrokerId (subscriptionGroupConfig.getWhichBrokerWhenConsumeSlowly ());} else {responseHeader.setSuggestWhichBrokerId (MixAll.MASTER_ID);} switch (this.brokerController.getMessageStoreConfig (). GetBrokerRole ()) {case ASYNC_MASTER: case SYNC_MASTER: break; case SLAVE: if (! this.brokerController.getBrokerConfig (). IsSlaveReadEnable ()) {response.setCode (ResponseCode.PULL_RETRY_IMMEDIATELY); responseHeader.setSuggestWhichBrokerId (MixAll.MASTER_ID);} break } if (this.brokerController.getBrokerConfig (). IsSlaveReadEnable ()) {/ / consume too slow, redirect to another machine if (getMessageResult.isSuggestPullingFromSlave ()) {responseHeader.setSuggestWhichBrokerId (subscriptionGroupConfig.getWhichBrokerWhenConsumeSlowly ());} / / consume ok else {responseHeader.setSuggestWhichBrokerId (subscriptionGroupConfig.getBrokerId ());}} else {responseHeader.setSuggestWhichBrokerId (MixAll.MASTER_ID);}

If it is found that the message accumulation on the master server exceeds 40% of the physical memory, the suggestWhichBrokerId will be set to the slave server broker ID.

There will also be a slaveReadEnable value to determine whether messages can be pulled from the server:

If slaveReadEnable=true and the accumulation has exceeded 40% of the physical memory, it is recommended to pull the message from the slave server, otherwise it is still pulled from the master server.

If slaveReadEnable=false, the messenger can only pull messages from the master server.

Org.apache.rocketmq.client.impl.consumer.PullAPIWrapper#updatePullFromWhichNode:

Public void updatePullFromWhichNode (final MessageQueue mq, final long brokerId) {AtomicLong suggest = this.pullFromWhichNodeTable.get (mq); if (null = = suggest) {this.pullFromWhichNodeTable.put (mq, new AtomicLong (brokerId));} else {suggest.set (brokerId);}}

When the consumer receives the data returned from the pull response, they will cache the brokerID that is recommended to be pulled next time. The next time the message is pulled, the brokerId will be pulled from the pullFromWhichNodeTable.

On the principle of RocketMQ master-slave read-write separation mechanism is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.