Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the installation and deployment of RocketMQ". In the daily operation, I believe many people have doubts about the installation and deployment of RocketMQ. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts about "the installation and deployment of RocketMQ". Next, please follow the editor to study!

Prepare for

Operating system: CentOS-6.9-x86_64-bin-DVD1.iso

Dependent environment: gcc 4.4.7.rar, jdk-8u121-linux-x64.tar.gz

Installation software: rocketmq-all-4.2.0.tar.gz

Deployment mode

The default RocketMQ deployment is cluster deployment, and business processing and storage are related to broker. RocketMQ also has a so-called master / slave node master/slave. The Slave here is not writable but readable, similar to Mysql master / slave mode.

It is recommended that when installing the system, when the disk is configured as RAID10, even if the machine is down and cannot be recovered, the messages will not be lost even if the disk and RAID10 are very reliable (a small amount of messages will be lost in asynchronous flushing, and none of them will be lost synchronously).

1. Single master node deployment (single Master)

This method is risky, because once the Broker is restarted or down, the entire service will be unavailable in a single-node deployment. It is * * not recommended for online * * environment.

2. Multi-master node deployment (multiple Master)

A cluster has no Slave, all Master, and can deploy 2 Master, 3 Master or even multiple N Master nodes. Compared with a single Master, its availability and stability are greatly improved. When a single Broker node is down or restarted, it will not affect the normal use of the entire cluster.

Messages on the down Broker node cannot be consumed immediately until the service is restored to normal state, and the real-time performance of messages will be affected.

3. Multi-master node and multi-slave node mode, asynchronous replication (multiple Master/Slave,ASYNC)

Each Master is configured with a Slave with multiple pairs of Master-Slave. HA is replicated asynchronously, and the master / slave has a short message delay of millisecond. Compared with multi-master deployment, its availability has been improved. Even if the disk is damaged, very few messages are lost, and the real-time performance of messages will not be affected, because after Master downtime, consumers can still consume from Slave, which is transparent to the application. No human intervention is required. The performance is almost the same as the multi-Master mode.

Master downtime, disk damage, a small number of messages will be lost

4. Multi-master node and multi-slave node mode, synchronous double write (multiple Master/Slave,SYNC)

Each Master is configured with a Slave, and there are multiple pairs of Master-Slave. HA uses synchronous double-write mode. Both master and slave are successfully written, and success is returned to the application. There is no single point for data and service. In the case of Master downtime, there is no message delay, and the service availability and data availability are very high.

The performance is slightly lower than the asynchronous replication mode, about 10% lower, and the RT for sending a single message is slightly higher. At present, after the main downtime, the standby cannot be automatically switched to the host, and the automatic switching function will be supported in the future.

Note: the above Broker and Slave pairs are paired by specifying the same brokerName parameters. The BrokerId of Master must be 0, and the BrokerId of Slave must be a number of large and 0. Multiple Slave can be mounted under another Maste. Multiple Slave under the same Master can be distinguished by specifying different BrokerId.

Dual master deployment 1, server environment serial number IP username password role mode 110.10.14.142root123456nameServer1 training brokerServer1Master1210.10.14.143root123456nameServer2Master2

Hosts add Information

IPNAME10.10.14.142nameServer110.10.14.142brokerServer110.10.14.143nameServer210.10.14.143brokerServer2

Modify the file vi / etc/hosts on two machines

2. Upload and extract RocketMQ files (two hosts)

Upload rocketmq-all-4.2.0.tar.gz file to / usr/local

Extract to / usr/local folder: tar-zxvf rocketmq-all-4.2.0.tar.gz-C / usr/local

The establishment of soft connection is convenient for operation.

Ln-s rocketmq-all-4.2.0 rocketmq

Create a storage path

# mkdir / usr/local/rocketmq/store# mkdir / usr/local/rocketmq/store/commitlog# mkdir / usr/local/rocketmq/store/consumequeue# mkdir / usr/local/rocketmq/store/index3, RocketMQ configuration file (two hosts)

View rocketmq/config

There are three folders, which exist by default, which means 2 master 2 slave asynchronous flushing disk, 2 master 2 slave synchronous flushing disk, and 2 master no slave node. What we deploy is the two masters. Modify the configuration file.

Modify broker-a.properties/broker-a.properties in the 2m-noslave folder

# brokerClusterName=rocketmq-cluster#broker name of the cluster to which it belongs. Note that different configuration files are filled in here. BrokerName=broker-a#0 indicates Master, > 0 indicates SlavebrokerId=0#nameServer address, and semicolon divides namesrvAddr= rocketmq-nameserver1:9876. When rocketmq-nameserver2:9876# sends messages, it automatically creates topic for which the server does not exist. Whether the number of queues created by default defaultTopicQueueNums=4# allows Broker to automatically create Topic, it is recommended that offline enable, online shutdown autoCreateTopicEnable=true# allows Broker to automatically create subscription groups, recommended offline enable, online shut down autoCreateSubscriptionGroup=true# Broker external service listening port listenPort=10911# delete file point, default 4: 00 am deleteWhen=04# file retention time Default 48-hour fileReservedTime=120#commitLog size per file default 1GmapedFileSizeCommitLog=1073741824#ConsumeQueue 30W per file by default Adjust mapedFileSizeConsumeQueue=300000#destroyMapedFileIntervalForcibly=120000#redeleteHangedFileInterval=120000# detection physical file disk space diskMaxUsedSpaceRatio=88# storage path storePathRootDir= / usr/local/rocketmq/store#commitLog storage path storePathCommitLog= / usr/local/rocketmq/store/commitlog# consumption queue storage path storage path storePathConsumeQueue=/usr/local/rocketmq/store/consumequeue# message index storage path storePathIndex=/usr/local/rocketmq/store/index#checkpoint file storage path storeCheckpoint=/usr/local/rocketmq/store/checkpoint# Abort file storage path abortFile=/usr/local/rocketmq/store/abort# limited message size maxMessageSize=65536#flushCommitLogLeastPages=4#flushConsumeQueueLeastPages=2#flushCommitLogThoroughInterval=10000#flushConsumeQueueThoroughInterval=60000#Broker role #-ASYNC_MASTER asynchronous replication Master#- SYNC_MASTER synchronous double write Master#- SLAVEbrokerRole=ASYNC_MASTER# flushing mode #-ASYNC_FLUSH asynchronous flushing disk #-SYNC_FLUSH synchronous flushing disk flushDiskType=ASYNC_FLUSH#checkTransactionMessageEnable=false# sending thread pool # number of sendMessageThreadPoolNums=128# pull message thread pool # pullMessageThreadPoolNums=1284, Modify the log configuration file (two hosts) mkdir-p / usr/local/rocketmq/logscd / usr/local/rocketmq/conf & & sed-I's modify ${user.home} # / usr/local/rocketmq#g' * .xml5, modify the startup script (in the development environment) (two hosts)

In general, our development environment is that the virtual machine does not have much memory and needs to modify its configuration file.

Vim / usr/local/rocketmq/bin/runbroker.sh

JAVA_OPT= "${JAVA_OPT}-server-Xms1g-Xmx1g-Xmn512m-XX:PermSize=128m-XX:MaxPermSize=320m"

Vim / usr/local/rocketmq/bin/runserver.sh

JAVA_OPT= "${JAVA_OPT}-server-Xms1g-Xmx1g-Xmn512m-XX:PermSize=128m-XX:MaxPermSize=320m"

When modifying the memory size of a java virtual machine, the heap memory must be greater than or equal to 1G, which is the minimum configuration requirement for starting the service.

6. Start NameServer (two hosts) cd / usr/local/rocketmq/binnohup sh mqnamesrv & 7, start BrokerServer

Start BrokerServer A [10.10.14.142]

Cd / usr/local/rocketmq/binnohup sh mqbroker-c / usr/local/rocketmq/conf/2m-noslave/broker-a.properties > / dev/null 2 > & 1 & View process jps View log file tail-f-n 500 / usr/local/rocketmq/logs/rocketmqlogs/broker.log

Start BrokerServer B [10.10.14.143]

Cd / usr/local/rocketmq/binnohup sh mqbroker-c / usr/local/rocketmq/conf/2m-noslave/broker-b.properties > / dev/null 2 > & 1 & View process jps View log file tail-f-n 500 / usr/local/rocketmq/logs/rocketmqlogs/broker.log8, Data cleaning cd / usr/local/rocketmq/binsh mqshutdown brokersh mqshutdown namesrv waiting to stop rm-rf / usr/local/rocketmq/storemkdir / usr/local/rocketmq/storemkdir / usr/local/rocketmq/store/commitlogmkdir / usr/local/rocketmq/store/consumequeuemkdir / usr/local/rocketmq/store/index follow the above steps to restart NameServer and BrokerServertail-f-n 500 / usr/local/rocketmq/logs/rocketmqlogs/broker.logtail-f-n 500 / usr/local/rocketmq/logs/rocketmqlogs/namesrv.log here The study on "how to install and deploy RocketMQ" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.