Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you about how to smoothly upgrade to master-slave switching in RocketMQ. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

1. Detailed explanation of core configuration parameters of RocketMQ DLedger multi-copy master-slave switch

The main configuration parameters are as follows:

Whether enableDLegerCommitLog enables DLedger, that is, whether RocketMQ master-slave switching is enabled. The default value is false. If master-slave switching needs to be turned on, this value needs to be set to true.

The raft group to which the dLegerGroup node belongs, which is recommended to be consistent with brokerName, such as broker-a.

For dLegerPeers cluster node information, the example configuration is as follows: N0-127.0.0.1 dledger 40911 position N1-127.0.0.1 dledger 40912 position N2-127.0.0.1 dledger 40913. Multiple nodes are separated by English colons, and a single entry follows the port: Port.

DLegerSelfId current node id. It is taken from the beginning of the entry in legerPeers, that is, N0 in the above example, and it is particularly important to emphasize that only the first character is English, and the other characters need to be configured as numbers.

The storage root directory for storePathRootDir DLedger log files, which is set to a different directory from storePathCommitLog in order to support smooth upgrades.

2. Build a master-slave synchronization environment

First of all, build a traditional master-slave synchronization architecture, pour a certain amount of data into the cluster, and then upgrade to the DLedger cluster.

I don't think it is very difficult to build a rocketmq master-slave synchronization cluster on the Linux server, so this article will not describe in detail according to the process, but only post the relevant configuration.

The deployment structure of the experimental environment is one master and one time, and the deployment diagram is as follows:

Next I will focus on posting the configuration file of broker. The broker configuration file on the 220 is as follows:

BrokerClusterName = DefaultClusterbrokerName = broker-abrokerId = 0deleteWhen = 04fileReservedTime = 48brokerRole = ASYNC_MASTERflushDiskType = ASYNC_FLUSHbrokerIP1=192.168.0.220brokerIP2=192.168.0.220namesrvAddr=192.168.0.221:9876;192.168.0.220:9876storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/storestorePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlogautoCreateTopicEnable=falseautoCreateSubscriptionGroup=false

The configuration file for broker on 221is as follows:

BrokerClusterName = DefaultClusterbrokerName = broker-abrokerId = 1deleteWhen = 04fileReservedTime = 48brokerRole = SLAVEflushDiskType = ASYNC_FLUSHbrokerIP1=192.168.0.221brokerIP2=192.168.0.221namesrvAddr=192.168.0.221:9876;192.168.0.220:9876storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/storestorePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlogautoCreateTopicEnable=falseautoCreateSubscriptionGroup=false

The relevant startup commands are as follows:

Nohup bin/mqnamesrv / dev/null 2 > & 1 & nohup bin/mqbroker-c conf/broker.conf / dev/null 2 > & 1 &

The cluster information after installation is shown in the figure:

3. Upgrade from master-slave synchronous cluster to DLedger3.1 deployment architecture

DLedger cluster requires at least 3 machines, so you need to introduce another machine to build DLedger. The deployment structure is as follows:

When upgrading from master-slave synchronous cluster to DLedger cluster, users are most concerned about whether the upgraded cluster is compatible with the original data, that is, whether the message can be stored in the message consumer side, or even queried. To facilitate subsequent verification, I first added a message to the mq cluster that is convenient to query (setting the key of the message) using the following program.

Public class Producer {public static void main (String [] args) throws MQClientException, InterruptedException {DefaultMQProducer producer = new DefaultMQProducer ("producer_dw_test"); producer.setNamesrvAddr ("192.168.0.220 args 9876"); producer.start (); for (int I = 600000; I)

< 600100; i ++) { try { Message msg = new Message("topic_dw_test_by_order_01",null , "m" + i,("Hello RocketMQ" + i ).getBytes(RemotingHelper.DEFAULT_CHARSET)); SendResult sendResult = producer.send(msg); //System.out.printf("%s%n", sendResult); } catch (Exception e) { e.printStackTrace(); Thread.sleep(1000); } } producer.shutdown(); System.out.println("end"); }} 消息的查询结果示例如下： 3.2 升级步骤 Step1：将 192.168.0.220 的 rocketmq 拷贝到 192.168.0.222，可以使用如下命令进行操作。在 192.168.0.220 上敲如下命令： scp -r rocketmq-all-4.5.2-bin-release/ root@192.168.0.222:/opt/application/rocketmq-all-4.5.2-bin-release >

Warm reminder: in the example, because the version is the same, in the actual process, the version needs to be upgraded, so you need to download the latest version first, and then copy the store directory in the old cluster to the store directory in the new cluster.

Step2: add dledger-related configuration properties to the broker.conf configuration files of the three servers in turn.

The 192.168.0.220 broker configuration file is as follows:

BrokerClusterName = DefaultClusterbrokerId = 0deleteWhen = 04fileReservedTime = 48brokerRole = ASYNC_MASTERflushDiskType = ASYNC_FLUSHbrokerIP1=192.168.0.220brokerIP2=192.168.0.220namesrvAddr=192.168.0.221:9876 192.168.0.220:9876storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/storestorePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlogautoCreateTopicEnable=falseautoCreateSubscriptionGroup=false# attributes related to dledger enableDLegerCommitLog=truestorePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_storedLegerGroup=broker-adLegerPeers=n0-192.168.0.220 enableDLegerCommitLog=truestorePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_storedLegerGroup=broker-adLegerPeers=n0 40911 tern 1-192.168.0.221 purl 40911 n2-192.168.0.222:40911dLegerSelfId=n0

The 192.168.0.221 broker configuration file is as follows:

BrokerClusterName = DefaultClusterbrokerName = broker-abrokerId = 1deleteWhen = 04fileReservedTime = 48brokerRole = SLAVEflushDiskType = ASYNC_FLUSHbrokerIP1=192.168.0.221brokerIP2=192.168.0.221namesrvAddr=192.168.0.221:9876 192.168.0.220:9876storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/storestorePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlogautoCreateTopicEnable=falseautoCreateSubscriptionGroup=false# configuration attributes related to dledger enableDLegerCommitLog=truestorePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_storedLegerGroup=broker-adLegerPeers=n0-192.168.0.220 purl 40911 n1-192.168.0.221 purl 40911 born2-192.168.0.222:40911dLegerSelfId=n1

The 192.168.0.222 broker configuration file is as follows:

BrokerClusterName = DefaultClusterbrokerName = broker-abrokerId = 0deleteWhen = 04fileReservedTime = 48brokerRole = ASYNC_MASTERflushDiskType = ASYNC_FLUSHbrokerIP1=192.168.0.222brokerIP2=192.168.0.222namesrvAddr=192.168.0.221:9876 192.168.0.220:9876storePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/storestorePathCommitLog=/opt/application/rocketmq-all-4.5.2-bin-release/store/commitlogautoCreateTopicEnable=falseautoCreateSubscriptionGroup=false# configuration related to dledger enableDLegerCommitLog=truestorePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_storedLegerGroup=broker-adLegerPeers=n0-192.168.0.220 enableDLegerCommitLog=truestorePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_storedLegerGroup=broker-adLegerPeers=n0 40911 tern 1-192.168.0.221 enableDLegerCommitLog=truestorePathRootDir=/opt/application/rocketmq-all-4.5.2-bin-release/store/dledger_storedLegerGroup=broker-adLegerPeers=n0 40911 Ten n2-192.168.0.222:40911dLegerSelfId=n2

> warm reminder: legerSelfId is n0, N1, N2 respectively. In a real production environment, storePathRootDir and storePathCommitLog in broker configuration files use separate root directories as far as possible, so that their disk utilization will not affect each other.

Step3: copy all the files under store/config to the congfig directory of dledger store.

Cd / opt/application/rocketmq-all-4.5.2-bin-release/store/cp config/* dledger_store/config/

> warm reminder: this step can be copied according to the directory configured at the time.

Step4: start three broker in turn.

Nohup bin/mqbroker-c conf/broker.conf / dev/null 2 > & 1 &

If the startup is successful, the cluster information seen in rocketmq-console is as follows:

3.3 verify message sending and message lookup

First of all, let's verify whether the message before the upgrade can be queried, then we still look for the message with a key of m600000. The result is shown in the figure:

Then let's test the message delivery. The test code is as follows:

Public class Producer {public static void main (String [] args) throws MQClientException, InterruptedException {DefaultMQProducer producer = new DefaultMQProducer ("producer_dw_test"); producer.setNamesrvAddr ("192.168.0.220 producer.start 9876"); producer.start (); for (int I = 600200; I < 600300) I + +) {try {Message msg = new Message ("topic_dw_test_by_order_01", null, "m" + I, ("Hello RocketMQ" + I) .getBytes (RemotingHelper.DEFAULT_CHARSET)); SendResult sendResult = producer.send (msg); System.out.printf ("% s% n", sendResult) } catch (Exception e) {e.printStackTrace (); Thread.sleep (1000);}} producer.shutdown (); System.out.println ("end");}}

The implementation results are as follows:

Go to the console to query the message, and the results also show that new messages can also be queried.

Finally, let's verify whether the sending of messages will be affected when the primary node is down.

In the process of sending the message, close the master node. The screenshot is as follows:

Let's take a look at the status of the cluster:

The above is how to smoothly upgrade to the master-slave switch in the RocketMQ shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.