Shulou(Shulou.com)06/02 Report--

This article shows you how to build a RabbitMQ cluster in docker. The content is concise and easy to understand. It will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Using docker to build RabbitMQ Cluster

The nodes in the RabbitMQ cluster contain memory nodes (RAM), disk nodes (Disk, message persistence), and at least one Disk node in the cluster. The cluster is divided into two kinds of cluster mode: normal mode and mirror mode.

Normal mode (default)

For normal mode, each node in the cluster has the same queue structure, but the message only exists in one node in the cluster. For consumers, if the message enters the Queue of node A, when pulled from node B, RabbitMQ will take the message out of An and send it to the consumer through B.

* * Application scenarios: * * this mode is more suitable for scenarios where messages need not be persisted, such as log queues. When the queue is not persistent and the node that created the queue is down, the client can reconnect to other nodes in the cluster and recreate the queue. If it is persistent, we can only wait for the failed node to recover. Disadvantages: unable to solve the single point of failure problem.

Mirror mode

When it is different from the normal mode, the message entity will actively synchronize at the mirror node instead of temporarily pulling data, which is highly available. In this mode, the mirror queue (mirror queue) has a set of election algorithms, that is, 1 master and n slaver. Requests from both producers and consumers will be forwarded to master.

* * Application scenarios: * * situations with high reliability requirements, such as placing orders and inventory queues. Disadvantages: if there are too many mirror queues and a large number of messages, the network bandwidth within the cluster will be consumed by this kind of synchronous communication.

Environment scaffolding creates three RabbitMQ nodes # creates RabbitMQ directories mkdir ~ / mydata/rabbitmqcd ~ / mydata/rabbitmq# creates three rabbitmq directories Store three node configuration information mkdir rabbitmq01 rabbitmq02 rabbitmq03# create rabbitmq01 node container docker run-d-- hostname rabbitmq01-- name rabbitmq01\-v ~ / mydata/rabbitmq/rabbitmq01:/var/lib/rabbitmq\-p 15672docker run 15672-p 5672docker run 5672\-e RABBITMQ_ERLANG_COOKIE='rabbitcookie' rabbitmq:management# create rabbitmq02 node container docker run-d-hostname rabbitmq02-- name rabbitmq02\-v / mydata/rabbitmq/rabbitmq02:/var/lib/rabbitmq\-p 15673pur15672- P 5673 link rabbitmq01:rabbitmq01 5672\-- link rabbitmq01:rabbitmq01\-e RABBITMQ_ERLANG_COOKIE='rabbitcookie' rabbitmq:management# create the rabbitmq03 node container docker run-d-- hostname rabbitmq03-- name rabbitmq03\-v ~ / mydata/rabbitmq/rabbitmq03:/var/lib/rabbitmq\-p 15674rabbitmq03 15672-p 5674rabbitmq03 5672\-- link rabbitmq01:rabbitmq01-- link rabbitmq02:rabbitmq02\-e RABBITMQ_ERLANG_COOKIE='rabbitcookie' rabbitmq:management## Note:-- hostname sets the container hostname RABBITMQ_ERLANG_COOKIE node authentication, cluster deployment needs to synchronize this value, and the value must be the same. # # use "--link" connection between multiple containers. This attribute cannot be reduced.

After the installation, I used http://192.168.0.100:15672 to access the test. The default account password is guest/guest, where 192.168.0.100 is my host IP. Displays the information that I have not joined any cluster, and I can only see the information of that node that I have visited.

Add the RabbitMQ node to cluster # enter the rabbitmq01 container and reinitialize it. If it is a new installation, reset can ignore the reset. Docker exec-it rabbitmq01 bashrabbitmqctl stop_apprabbitmqctl resetrabbitmqctl start_appexit# enters the rabbitmq02 container, reinitializes it, and adds the 02 node to the cluster docker exec-it rabbitmq02 bashrabbitmqctl stop_apprabbitmqctl resetrabbitmqctl join_cluster-- ram rabbit@rabbitmq01 # parameter "--ram" indicates that it is set to the memory node, ignoring this parameter defaults to the disk node. Rabbitmqctl start_appexit# enters the rabbitmq03 container, reinitializes it, and adds 03 nodes to the cluster docker exec-it rabbitmq03 bashrabbitmqctl stop_apprabbitmqctl resetrabbitmqctl join_cluster-ram rabbit@rabbitmq01rabbitmqctl start_appexit cluster test access

Once set up, I access it using http://192.168.0.100:15672. The default account password is guest/guest, where 192.168.0.100 is my host IP. Three nodes have been started, and so far, we have completed the establishment of the RabbitMQ normal mode cluster, starting three nodes, one disk node and two memory nodes. But if the disk node dies, the data is lost. Therefore, our cluster scheme needs to be further transformed into a mirror mode cluster.

Transformation of cluster in mirror mode

Mirror queue is a new feature brought by Rabbit2.6.0 version, which allows built-in dual-active redundancy option. Unlike ordinary queues, mirror nodes in other nodes in the cluster have copies of slave queues. Once the master node is not available, the oldest slave queue will be elected as the new master queue.

* * how the mirror queue works: * * to some extent you can think of the mirror queue as having a hidden fanout switch that instructs the channel to distribute messages to the slave queue.

Set up the mirror queue

Set the mirror queue command: "rabbitmqctl set_policy name matching mode (regular) mirror definition". For example, set a mirror queue named ha, and the command to match all queues whose names begin with amp is stored on two nodes:

# enter any container docker exec-it rabbitmq01 bash# to set policy matching all queues whose names begin with amp are stored on two nodes as follows: rabbitmqctl set_policy-p / ha "^ amp*"'{"ha-mode": "exactly", "ha-params": 2}'# or # configure queues with all names for high availability configuration rabbitmqctl set_policy-p / ha "^'{" ha-mode ":" all " "ha-sync-mode": "automatic"}'# query policy rabbitmqctl list_policies-p / # View all policies under vhost (policies)

You can see that the mirror queue is set up with a total of four parameters, each separated by spaces.

Parameter 1: policy name, which can be filled in freely. In addition, we name it ha (highly available)

Reference 2:-p / to set which virtual host, you can use rabbitmqctl list_policies-p / to view all the policies under vhost (policies).

Parameter 3: matching rules for queue names, expressed by regular expressions

Parameter 4: the principal rule of the image queue, which is a json string, and is divided into three attributes: ha-mode | ha-params | ha-sync-mode. They are explained as follows:

Ha-mode: mirror mode, classification: all/exactly/nodes,all is stored on all nodes; exactly stores x nodes, and the number of nodes is specified by ha-params; nodes specifies the name on the stored node, specified by ha-params

Ha-params: as a parameter, as a supplement to ha-mode

Ha-sync-mode: mirror message synchronization method: automatic (automatic), manually (manual)

Now that the rabbitmq cluster has been built, visit and create a queue to test it! As shown in the picture!

View image queues rabbitmqctl list_policies delete mirror queues rabbitmqctl clear_policy summary

You can try it yourself in the docker container. Finally, you don't have to build it yourself when you use it in the production environment. At present, Aliyun and Tencent Cloud are also improving the excellent and mature out-of-the-box messaging middleware for everyone to choose from, such as RocketMQ, Kafka and RabbitMQ.

Aliyun message queue MQ message queue (Message Queue, referred to as MQ) is the infrastructure for building distributed Internet applications. The loosely coupled architecture design through MQ can improve system availability and scalability, and is the best design scheme for modern applications. If there is a message queue RocketMQ version of Alibaba officially designated message products, mature, stable and advanced technical system to create financial-level message services, feel the perfect experience of Singles Day products.

Tencent Cloud message queuing TDMQ (Tencent Distributed Message Queue, hereinafter referred to as TDMQ) is a financial-level distributed message middleware developed by Pulsar, a top open source project in Apache. Its architecture design of the separation of computing and storage makes it have excellent cloud native and Serverless features, users use according to quantity, do not need to care about the underlying resources. It has native Java, C++, Python, GO and other API, and supports kafka protocol and HTTP protocol access at the same time. It can provide distributed application systems with the ability of asynchronous decoupling, peak cutting and valley filling, and has the characteristics of massive message accumulation, high throughput and reliable retry needed by Internet applications. TDMQ has been applied to most of Tencent's billing scenarios, including main payment path, real-time reconciliation, real-time monitoring, real-time analysis of big data, and so on.

Other questions

If there is a change expiration warning: RABBITMQ_ERLANG_COOKIE env variable support is deprecated and will be REMOVED in a future version. Use the $HOME/.erlang.cookie file or the-- erlang-cookie switch instead.

Configure the same Erlang Cookie

In some special cases, such as several individual physical machines that have been running for some time, we have not set the same Erlang cookie value before. now we want to deploy a single physical machine into a cluster to realize that we need to synchronize the cookie value of Erlang.

1. Why configure the same erlang cookie?

Because RabbitMQ is implemented in Erlang, Erlang Cookie is equivalent to the secret key of communication between different nodes, and Erlang nodes are authenticated by exchanging Erlang Cookie.

Location of 2.Erlang Cookie

To know the location of the Erlang Cookie, first get the home dir path in the RabbitMQ startup log as the root path. Use: "docker logs Container name" to view, as shown below:

So the entire path to Erlang Cookie is "/ var/lib/rabbitmq/.erlang.cookie".

Note: everyone's erlang cookie location may be different, be sure to check your own home dir path.

3. Copy Erlang Cookie to other RabbitMQ nodes

After you get the Erlang Cookie of the first RabbitMQ, you just need to copy the file to another RabbitMQ node.

The copy command between the physical machine and the container is as follows:

Copy files from container to physical machine: docker cp container name: container directory physical machine directory

Copy files from physical machine to container: docker cp physical machine directory container name: container directory

Set Erlang Cookie file permissions: "chmod 600 / var/lib/rabbitmq/.erlang.cookie".

The above content is how to build a RabbitMQ cluster in docker. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.