Shulou(Shulou.com)06/01 Report--

This article will explain in detail about the high availability principle and actual deployment of RabbitMQ cluster. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

Today, let's talk about the principle and deployment process of RabbitMQ cluster.

I. introduction

In previous articles, we described in detail the internal structure and use of RabbitMQ, as well as the integration of SpringBoot and RabbitMQ, which are based on a single RabbitMQ.

We know that in the current popularity of micro services, once a single server is down, it is basically impossible to provide highly available services, so in order to ensure the high availability of services, we usually build a RabbitMQ cluster in the production environment. Even if a RabbitMQ fails, other normal RabbitMQ servers can still be used, and the continuous operation of applications will not be affected.

Second, the principle of cluster architecture

In previous articles, we introduced that there are various basic artifacts within RabbitMQ, including queues, switches, bindings, virtual hosts, etc., which form the basis of AMQP protocol message communication, and these components exist in the form of metadata, which is always recorded inside the RabbitMQ. They are:

Queue metadata: queue names and their properties switch metadata: switch name, type, and attribute binding metadata: a simple table shows how messages are routed to queue vhost metadata: provides namespaces and security attributes for queues, switches, and bindings within vhost

This metadata is essentially a query table, which includes the name of the switch and the binding list of a queue. When you publish the message to the switch, you actually match the routing key on the message with the binding list of the switch, and then route the message out.

Message routing table

With this mechanism, it is much easier to pass switch messages on all nodes, and what RabbitMQ does is copy the switch metadata to all nodes, so that every channel on each node can access the complete switch.

If the message producer is connected to node 2 or node 3, and the complete data of queue 1 is not on the two nodes, then the two nodes mainly play a routing and forwarding role in the process of sending messages. according to the metadata on the two nodes is forwarded to node 1, the final message will be stored on queue 1 of node 1.

Similarly, if the message consumer is connected to node 2 or node 3, then these two nodes will also act as routing nodes and will pull messages from queue 1 of node 1 for consumption.

What is different from the common cluster master-slave architecture mode is that in RabbitMQ cluster mode, it is only synchronous metadata, and each queue content is still on its own server node.

This design is mainly based on the performance and storage space of the cluster itself:

Storage space: the real place to store data is in the queue. If each cluster node has complete data copies of all queues, then the storage space of each node will be very large, and the message backlog capacity of the cluster will be very weak. For example, if you now store 3G queue content, there will be insufficient memory space on another node with only 1G storage space, that is, the message backlog capacity cannot be improved through the expansion of cluster nodes. Performance: the publisher of the message needs to copy the message to each cluster node, and each message will trigger disk activity, which will lead to a sharp increase in performance load throughout the cluster.

Since the content of each queue is still on its own server node, it also brings a new problem, that is, if the server where the queue is located is down, is all the queue data on the existing server lost?

There are two scenarios for RabbitMQ to store data on a single node:

Memory mode: this mode stores data in memory, and if the server suddenly goes down and restarts, the queues attached to the node and their associated bindings are lost, and consumers can reconnect to the cluster and recreate the queue Disk mode: this mode stores the data on the disk. If the server suddenly goes down and restarts, the data will automatically recover, the queue can transfer data again, and consumers cannot reconnect to the cluster and recreate the queue on other nodes until the failed disk node is restored. If consumers continue to declare the queue on other nodes, they will get a 404 NOT_FOUND error. This ensures that when the failed node is restored and joins the cluster, the queue messages on the node will not be lost, and the problem of redundancy of the queue in more than one node can be avoided.

Each node in the cluster is either a memory node or a disk node. If it is a memory node, all metadata information is stored in memory only, while the disk node not only stores all metadata in memory, but also persists it to disk.

On a single-node RabbitMQ, only that node is allowed to be a disk node, which ensures that all configuration and metadata information about the system is restored from disk after the node fails or restarts the node.

On the RabbitMQ cluster, there is at least one disk node, that is, two or more disk nodes need to be added in the cluster environment, so that one of them fails and the cluster can still keep running. Other nodes are set as memory nodes, which makes operations such as queue and switch declarations faster and metadata synchronization more efficient.

III. Cluster deployment

In order to be consistent with the production environment, we choose the CentOS7 operating system to deploy the environment and create three virtual machines respectively.

# IP of 3 servers

197.168.24.206

197.168.24.233

197.168.24.234

Release the firewall restrictions to ensure that the three server networks can be interconnected!

3.1. Reset hostname

Since the RabbitMQ cluster connection connects to the service through the hostname, you must ensure that each hostname can be ping connected, and reset the hostnames of the three servers, so you need to do the following:

# modify the hostname of Node 1

Hostname node1

# modify the hostname of node 2

Hostname node2

# modify the hostname of node 3

Hostname node3

Edit the / etc/hosts file to add the following to the / etc/hosts of the three machines:

Sudo vim / etc/hosts

The content is added as follows:

197.168.24.206 node1

197.168.24.233 node2

197.168.24.234 node3

3.2. rabbitMQ installation

RabbitMQ communicates based on erlang, compared with other software, installation is troublesome, but this example uses rpm installation, any novice can complete the installation, the process is as follows!

3.2.1. Pre-installation command preparation

Enter the following command to complete the preparation of the environment before installation.

Yum install lsof build-essential openssl openssl-devel unixODBC unixODBC-devel make gcc gcc-c++ kernel-devel m4 ncurses-devel tk tc xz wget vim

3.2.2. Download the installation packages for RabbitMQ, erlang and socat

This download is the RabbitMQ-3.6.5 version, using rpm one-click installation, suitable for beginners to start directly.

First create a rabbitmq directory, the directory path in this example is / usr/app/rabbitmq, and then execute the following command under the directory to download the installation package!

Download erlangwget www.rabbitmq.com/releases/erlang/erlang-18.3-1.el7.centos.x86_64.rpm

Download socatwget http://repo.iotti.biz/CentOS/7/x86_64/socat-1.7.3.2-5.el7.lux.x86_64.rpm

Download rabbitMQwget www.rabbitmq.com/releases/rabbitmq-server/v3.6.5/rabbitmq-server-3.6.5-1.noarch.rpm

The final directory file is as follows:

3.2.3. Install the software package

After downloading, install the software packages sequentially, which is very important.

Install erlangrpm-ivh erlang-18.3-1.el7.centos.x86_64.rpm

Install socatrpm-ivh socat-1.7.3.2-5.el7.lux.x86_64.rpm

Install rabbitmqrpm-ivh rabbitmq-server-3.6.5-1.noarch.rpm

After the installation is complete, modify the configuration of rabbitmq so that the default configuration file is in the / usr/lib/rabbitmq/lib/rabbitmq_server-3.6.5/ebin directory.

Vim / usr/lib/rabbitmq/lib/rabbitmq_server-3.6.5/ebin/rabbit.app

Modify the value of the loopback_users node!

Re-order the rabbit node name separately

Vim / etc/rabbitmq/rabbitmq-env.conf

Add a line to the file, as follows!

NODENAME=rabbit@node1

The other two node commands are similar, and then save! Start the service with the following command!

# start the service

Rabbitmq-server start &

# stop the service

Rabbitmqctl stop

Query whether the service has been started successfully with the following command!

Lsof-iRU 5672

If 5672 has been monitored, it has been started successfully!

3.2.4. Start the visual control console

Enter the following command to start the console!

Rabbitmq-plugins enable rabbitmq_management

Open http://ip:15672 with a browser. The ip here is the ip of the CentOS system. The result is as follows:

The account and password defaults to guest. If you cannot access it, check whether the firewall is enabled, and turn it off if it is enabled.

The monitoring platform after login has the following interface:

3. Copy Erlang cookie

In RabbitMQ cluster environment, metadata synchronization is based on cookie sharing scheme.

Here, copy the cookie file of node1 to node2. Because the permission of this file is 400 to facilitate transmission, modify the permission first, not necessary operation, so you need to modify the permission of the file in node1 to 777

Chmod 777 / var/lib/rabbitmq/.erlang.cookie

Copy to node 2 with scp, and node 3 does the same.

Scp / var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/

Finally, change the permissions back.

Chmod 400 / var/lib/rabbitmq/.erlang.cookie

3.4. Form a cluster

Execute the following command on node 2:

# stop rabbitmq service

Rabbitmqctl stop_app

# clear node status

Rabbitmqctl reset

# node2 and node1 form a cluster. Node2 must be able to ping through the hostname of node1

Rabbitmqctl join_cluster rabbit@node1

# enable rabbitmq service

Rabbitmqctl start_app

The operation of Node 3 is similar!

Check the cluster status on any machine:

Rabbitmqctl cluster_status

The first line: represents the current node information. The second line: represents the node members in the cluster, and disc indicates that these are all disk nodes. The third line: represents the running node members.

Login to the visual console, you can clearly see that the three service nodes have been related to each other!

If you want to remove a node from the cluster, take removing node 3 as an example, you can do this as follows!

# first stop the node service to be removed

Rabbitmqctl stop

# remove Node 3

Rabbitmqctl-n rabbit@node1 forget_cluster_node rabbit@node3

If you cannot start rabbitMQ after removal, delete the existing mnesia information!

Rm-rf / var/lib/rabbitmq/mnesia

Then restart the service again!

3.5. Set the memory node # set the node as the memory node when joining (disk node by default)

Rabbitmqctl join_cluster rabbit@node1-ram

Where-ram refers to as a memory node, if not added, it defaults to the disk node.

If the node is already a disk node in the cluster, you can change the node to a memory node with the following command:

# stop rabbitmq service

Rabbitmqctl stop_app

# change a node to a memory node

Rabbitmqctl change_cluster_node_type ram

# enable rabbitmq service

Rabbitmqctl start_app

3.6, Mirror queue

As mentioned above, by default, the queue is saved on only one of the nodes. When a node fails, although all metadata information can be recovered from the disk node to this node, the queue message content of the memory node is not good, which will lead to the loss of the message.

RabbitMQ was aware of this problem a long time ago and added the queue redundancy option: mirrored queues in versions later than 2.6.

The so-called mirror queue means that the master queue (master) still exists on only one node. Through the associated rabbitMQ server, the master queue synchronizes messages to each node, that is, the so-called master-slave mode, to back up the messages of the master queue.

If the master queue does not fail, then its workflow is the same as the normal queue, producers and consumers will not be aware of its change, and when the message is published, it is still routed to the master queue, while the master queue spreads and synchronizes the message to the rest of the slave queue through a broadcast-like mechanism, which is a bit like a fanout switch. Consumers still read messages from the home queue.

Once the master queue fails, the cluster will elect the new master queue from the oldest slave queue, which achieves the high availability of the queue, but we must not abuse this mechanism, as mentioned above, the redundant operation of queues can lead to the inability to increase storage space by expanding nodes, and can cause performance bottlenecks.

The format of the command is as follows:

Rabbitmqctl set_policy [- p Vhost] Name Pattern Definition [Priority]

Parameter description:

-p Vhost: optional parameter, which is set for the queue under the specified vhost

Name: the name of the policy

Pattern: matching pattern of queue (regular expression)

Definition: image definition, including three parts: ha-mode, ha-params, and ha-sync-mode

Ha-mode: indicates the mode of the mirror queue. Valid value is all/exactly/nodes.

All: indicates mirroring on all nodes in the cluster

Exactly: indicates that the mirror is performed on a specified number of nodes, the number of which is specified by ha-params

Nodes: indicates that it is mirrored on the specified node, and the node name is specified by ha-params

Ha-params: parameters to be used in ha-mode mode

Ha-sync-mode: synchronizes messages in the queue. Valid values are automatic and manual

Priority: optional parameter, priority of policy

For example, declare a policy named ha-all that matches a queue whose name begins with ha and configures the mirror to all nodes in the cluster:

Rabbitmqctl set_policy ha-all "^"'{"ha-mode": "all"}'

There are many similar operations. For more information, please see the official api.

IV. Load balancing of clusters

HAProxy provides high availability, load balancing, agents based on TCP and HTTP applications, and supports virtual hosts. It is a free, fast and reliable solution. According to official data, its maximum limit supports 10G concurrency. HAProxy supports network switching from layer 4 to layer 7, that is, covering all TCP protocols. That is, Haproxy even supports load balancing for Mysql. To achieve soft load balancing for RabbitMQ clusters, you can choose HAProxy here.

4.1By HAProxy installation

The installation of HAProxy is also very simple, deployed on a separate server, and can be installed with the following command!

Yum install haproxy

Edit the HAProxy configuration file:

Vim / etc/haproxy/haproxy.cfg

We just need to add the following configuration at the end of the file!

# binding configuration

Listen rabbitmq_cluster

Bind 0.0.0.0:5672

# configure TCP mode

Mode tcp

# weighted polling

Balance roundrobin

# RabbitMQ Cluster Node configuration

Server rmq_node1 197.168.24.206:5672 check inter 5000 rise 2 fall 3 weight 1

Server rmq_node2 197.168.24.233:5672 check inter 5000 rise 2 fall 3 weight 1

Server rmq_node3 197.168.24.234:5672 check inter 5000 rise 2 fall 3 weight 1

# address of haproxy monitoring page

Listen monitor

Bind 0.0.0.0:8100

Mode http

Option httplog

Stats enable

Stats uri / stats

Stats refresh 5s

Binding configuration parameters description:

Bind: the client connection IP address and port number are defined here, which are used for the client connection balance roundrobin: represents the weighted polling load balancing algorithm

RabbitMQ cluster node configuration instructions:

Server rmq_node1: defines the identity of the RabbitMQ service in the HAProxy 197.168.24.206 RabbitMQ 5672: identifies the service address of the backend RabbitMQ check inter 5000: indicates how many milliseconds to check whether the RabbitMQ service is available. The example parameter value is 5000rise 2: indicates the number of health checks required by the RabbitMQ service before it can be reconfirmed after a failure. The example parameter value is 2fall 2: indicates how many failed health checks are required. HAProxy will stop using this RabbitMQ service. The example parameter value is 2weight 1: indicates the weight ratio. The lower the value is, the data allocation will be given priority. The example parameter value is 1.

Start HAProxy:

/ usr/sbin/haproxy-f / etc/haproxy/haproxy.cfg

Log in to the http://ip:8100/statsweb management interface to monitor and view!

5. Use of Java client

If the HAProxy proxy server is configured, you can use the HAProxy proxy server address directly!

/ / ConnectionFactory creates a physical connection to MQ

ConnectionFactory = new ConnectionFactory ()

ConnectionFactory.setHost ("197.168.24.207"); / / proxy server address

ConnectionFactory.setPort (5672); / / proxy server port

ConnectionFactory.setUsername ("admin"); / / guest can only be accessed locally, and the user needs to be re-established when sending messages through the proxy server

ConnectionFactory.setPassword ("admin"); / / guest

ConnectionFactory.setVirtualHost ("/"); / / virtual host

If there is no proxy server, use the CachingConnectionFactory class of Spring for configuration.

Take the SpringBoot project as an example, the configuration file is as follows:

Spring.rabbitmq.addresses=197.168.24.206:5672197.168.24.233:5672197.168.24.234:5672

Spring.rabbitmq.username=guest

Spring.rabbitmq.password=guest

Spring.rabbitmq.virtual-host=/

The RabbitConfig configuration classes are as follows:

@ Configuration

Public class RabbitConfig {

/ * *

* initialize the connection factory

* @ param addresses

* @ param userName

* @ param password

* @ param vhost

* @ return

, /

@ Bean

ConnectionFactory connectionFactory (@ Value ("${spring.rabbitmq.addresses}") String addresses

@ Value ("${spring.rabbitmq.username}") String userName

@ Value ("${spring.rabbitmq.password}") String password

@ Value ("${spring.rabbitmq.virtual-host}") String vhost) {

CachingConnectionFactory connectionFactory = new CachingConnectionFactory ()

ConnectionFactory.setAddresses (addresses)

ConnectionFactory.setUsername (userName)

ConnectionFactory.setPassword (password)

ConnectionFactory.setVirtualHost (vhost)

Return connectionFactory

}

/ * *

* re-instantiate RabbitAdmin operation classes

* @ param connectionFactory

* @ return

, /

@ Bean

Public RabbitAdmin rabbitAdmin (ConnectionFactory connectionFactory) {

Return new RabbitAdmin (connectionFactory)

}

/ * *

* re-instantiate RabbitTemplate operation classes

* @ param connectionFactory

* @ return

, /

@ Bean

Public RabbitTemplate rabbitTemplate (ConnectionFactory connectionFactory) {

RabbitTemplate rabbitTemplate=new RabbitTemplate (connectionFactory)

/ / data is converted to json and stored in message queue

RabbitTemplate.setMessageConverter (new Jackson2JsonMessageConverter ())

Return rabbitTemplate

}

} this is the end of sharing about the high availability principle and actual deployment of RabbitMQ cluster. I hope the above content can be helpful to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.