Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of RabbitMQ cluster architecture, which has a certain reference value. Interested friends can refer to it. I hope you will gain a lot after reading this article.

First, why use clusters?

As one of the best features of RabbitMQ, built-in cluster has two functions:

Allow consumers and producers to continue to run if the Rabbit node crashes

Expand Rabbit to handle more messages and carry more traffic by adding more nodes

Second, the characteristics of the cluster

RabbitMQ's cluster is made up of multiple nodes, but we find that not every node has a full copy of all the queues.

RabbitMQ node incomplete copy feature

Why doesn't RabbitMQ copy all queue contents and states to all nodes by default?

There are two reasons:

Storage space-if each node has a full copy of all queues, instead of adding new storage space, the new node adds more redundant data.

Performance-if the message needs to be safely copied to each cluster node, the new node will increase the network and disk load, which goes against the original intention of establishing the cluster, and the new node does not improve the ability to process messages. At best, it can maintain the same performance as a single node, or even worse.

So the other non-owner nodes only know the metadata of the queue and the pointer to the queue node.

Third, cluster exception handling

According to the characteristic that the node is not without secure copy, when the cluster node crashes, the node queue and associated bindings are lost, and the consumers attached to the queue lose their subscription information, so how to deal with this problem?

This problem can be divided into two situations:

The message has been persisted, so when the node recovers, the message is restored.

The message is not persistent, so you can use the dual active redundancy queue described below. The mirror queue ensures the reliability of the message.

IV. Cluster node types

There are two types of storage for nodes:

Disk node

Memory node

The disk node is that the configuration information and meta-information are stored on the disk, and the internal secondary node stores these information in memory. Of course, the performance of the internal secondary node is much better than that of the disk node.

The single-node system must be a disk node, otherwise all system configuration information will be lost every time you restart RabbitMQ.

RabbitMQ requires that there is at least one disk node in the cluster and must notify the disk node when the node joins and leaves the cluster.

Special exception: the only disk node in the cluster crashes

What happens if the only disk node in the cluster crashes as a result?

If the disk node of the only disk crashes, you cannot do the following:

Cannot create queue

Cannot create switch

Cannot create binding

Cannot add user

Permissions cannot be changed

Cannot add or delete clusters

Summary: if the disk node of the only disk crashes, the cluster can stay running, but you can't change anything.

Solution: set up two disk nodes in the cluster, as long as one can, you can operate normally.

Fifth, the method of building cluster

In this chapter, we use Docker to create a RabbitMQ cluster, first, because it is easy to operate, second, because it can make more use of server hardware resources, and third, Docker is also the mainstream deployment solution now. For more details on Docker, please see my article: "deploy RabbitMQ clusters with Docker." next, let's go to our body. Cluster building is divided into two steps:

Step 1: install multiple RabbitMQ

Step 2: join the RabbitMQ node to the cluster

Step 1: install multiple RabbitMQdocker run-d-- hostname rabbit1-- name myrabbit1-p 15672-p 5672-- 5672-- e RABBITMQ_ERLANG_COOKIE='rabbitcookie' rabbitmq:3.6.15-managementdocker run-d-- hostname rabbit2-- name myrabbit2-p 5673-- link myrabbit1:rabbit1-e RABBITMQ_ERLANG_COOKIE='rabbitcookie' rabbitmq:3.6.15-managementdocker run-d-- hostname rabbit3-- name myrabbit3-p 5674V 5672-- link myrabbit1:rabbit1-- link myrabbit2:rabbit2-e RABBITMQ_ERLANG_COOKIE='rabbitcookie' rabbitmq:3.6.15-management

For the meaning of the parameters, see the section "starting RabbitMQ" above.

Note:

Use "--link" connection between multiple containers. This property cannot be reduced.

The Erlang Cookie value must be the same, that is, the value of the RABBITMQ_ERLANG_COOKIE parameter must be the same, as shown in the section "configuring the same Erlang Cookie" below.

Step 2: join the RabbitMQ node to the cluster

Set up Node 1:

Docker exec-it myrabbit1 bash

Rabbitmqctl stop_app

Rabbitmqctl reset

Rabbitmqctl start_app

Exit

Set node 2 and join the cluster:

Docker exec-it myrabbit2 bash

Rabbitmqctl stop_app

Rabbitmqctl reset

Rabbitmqctl join_cluster-ram rabbit@rabbit1

Rabbitmqctl start_app

Exit

The parameter "--ram" indicates that it is set to a memory node, while ignoring the secondary parameter defaults to the disk node.

Set node 3 to join the cluster:

Docker exec-it myrabbit3 bash

Rabbitmqctl stop_app

Rabbitmqctl reset

Rabbitmqctl join_cluster-ram rabbit@rabbit1

Rabbitmqctl start_app

Exit

Once set up, the http:// physical machine ip:15672 is used to access it. The default account password is guest/guest, as shown below:

So far, we have completed the establishment of the RabbitMQ cluster, starting 3 nodes, 1 disk node and 2 memory nodes.

Set the node type

If you want to change the node type, you can change it by command, as follows:

Rabbitmqctl stop_app

Rabbitmqctl change_cluster_node_type dist

Rabbitmqctl change_cluster_node_type ram

Rabbitmqctl start_app

Remove nod

If you want to remove the node from the cluster, you can use the following command:

Rabbitmqctl stop_app

Rabbitmqctl restart

Rabbitmqctl start_app

Cluster restart sequence

The order in which the cluster is restarted is fixed and reversed. As follows:

Boot sequence: disk node = > memory node

Closing order: memory node = > disk node

The final shutdown must be a disk node, otherwise it may cause abnormal situations such as cluster startup failure, data loss and so on.

VI. Mirror queue

Mirror queue is a new feature brought by Rabbit2.6.0 version, which allows built-in dual-active redundancy option. Unlike ordinary queues, mirror nodes in other nodes in the cluster have copies of slave queues. Once the master node is not available, the oldest slave queue will be elected as the new master queue.

How mirrored queues work: to some extent you can think of mirrored queues as having a hidden fanout switch that instructs the channel to distribute messages to the slave queue.

Set up the mirror queue

Set the mirror queue command: "rabbitmqctl set_policy name matching mode (regular) mirror definition". For example, set a mirror queue named mypolicy, and the command to match all queues whose names begin with amp is stored on two nodes:

Rabbitmqctl set_policy mypolicy "^ amp*"'{"ha-mode": "exactly", "ha-params": 2}'

You can see that the mirror queue is set up with a total of three parameters, each of which is separated by spaces.

Parameter 1: name, which can be entered at will

Parameter 2: matching rules for queue names, expressed by regular expressions

Parameter 3: the principal rule of the image queue, which is a json string, and is divided into three attributes: ha-mode | ha-params | ha-sync-mode. They are explained as follows:

Ha-mode: mirror mode, classification: all/exactly/nodes,all is stored on all nodes; exactly stores x nodes, and the number of nodes is specified by ha-params; nodes specifies the name on the stored node, specified by ha-params

Ha-params: as a parameter, as a supplement to ha-mode

Ha-sync-mode: mirror message synchronization method: automatic (automatic), manually (manual)

The effect of setting the image queue to store 2 nodes is as follows:

View the mirror queue

Rabbitmqctl list_policies

Delete Mirror queue

Rabbitmqctl clear_policy

Thank you for reading this article carefully. I hope the article "sample Analysis of RabbitMQ Cluster Architecture" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.