Shulou(Shulou.com)06/01 Report--

This article mainly shows you "RabbitMQ how to achieve cluster management", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "RabbitMQ how to achieve cluster management" this article.

The principle of RabbitMQ Cluster Scheme

RabbitMQ, a message queuing middleware product, is written based on Erlang, and the Erlang language is inherently distributed (realized by synchronizing the magic cookie of each node of the Erlang cluster). Therefore, RabbitMQ naturally supports Clustering. This makes it unnecessary for RabbitMQ itself to implement the HA scheme and save the cluster metadata through ZooKeeper, as ActiveMQ and Kafka do. Clustering is a way to ensure reliability, and it can be scaled horizontally to increase message throughput. Let's take a look at the overall solution of the RabbitMQ cluster:

The above figure uses three nodes to form a cluster of RabbitMQ. The metadata information of Exchange A (switch, children's shoes that are not clear about the basic concept of RabbitMQ) is consistent on all nodes, while the complete data of Queue (the queue where messages are stored) will only exist on the node it created. The other nodes only know the metadata information for this queue and a pointer to the owner node for queue.

Synchronization of RabbitMQ Cluster metadata

The RabbitMQ cluster always synchronizes four types of internal metadata (similar to indexes): a. Queue metadata: queue name and its properties; b. Switch metadata: switch name, type, and attributes; c. Bind metadata: a simple table shows how to route messages to queues; d.vhost metadata: provides namespaces and security attributes for queues, exchanges, and bindings within vhost; therefore, when a user accesses any of these RabbitMQ nodes, the queue/user/exchange/vhost and other information queried through rabbitmqctl is the same.

Why RabbitMQ clusters only use metadata synchronization

I'm sure many students will ask, if you want to implement the HA solution, why not just keep a copy of the complete data of all the Queue in the RabbitMQ cluster on all nodes? (it can be similar to the main main mode of MySQL) in this way, when any node fails or becomes unavailable, the client of the user can publish and subscribe to the message as long as it can connect to other nodes. I think the design of RabbitMQ is mainly based on the performance and storage space of the cluster itself. First, storage space. If each cluster node has full data copies of all Queue, then the storage space of each node will be very large, and the message backlog capacity of the cluster will be very weak (the message backlog capacity cannot be improved through the expansion of cluster nodes). Second, performance, the publisher of the message needs to copy the message to each cluster node, and the overhead of network and disk synchronous replication will increase significantly for persistent messages.

The basic principle of sending / subscribing messages in RabbitMQ Cluster

The working schematic diagram of the RabbitMQ cluster is as follows:

Scenario 1. The client connects directly to the node where the queue is located.

If a message producer or message consumer publishes or subscribes messages to node 1 through an amqp-client client connection, then the sending and receiving of messages in the cluster is only related to node 1, which is no problem; what if the client is connected to node 2 or node 3 (queue 1 data is not on that node)?

Scenario 2. The client connects to the node where the non-queue data resides.

If the message producer is connected to node 2 or node 3, and the complete data of queue 1 is not on the two nodes, then these two nodes mainly play a routing and forwarding role in the process of sending messages, and according to the metadata on these two nodes (that is, the pointer to the owner node to queue) is forwarded to node 1, the final message will be stored on queue 1 of node 1. Similarly, if the message consumer is connected to node 2 or node 3, then these two nodes will also act as routing nodes and will pull messages from queue 1 of node 1 for consumption.

What is the difference between declaring queue, exchange, and doing binding in a single node system and a multi-node cluster system?

When you declare queue on a single node, you will get a Queue.Declare-ok response as long as the relevant metadata on the node is changed, while declaring queue on cluster requires that all node on the cluster must be successfully updated with metadata before you can get a Queue.Declare-ok response. In addition, if the node type is RAM node, the changed data is saved only in memory, and if the type is disk node, the data saved on disk is also changed.

Does the client connect to any node in cluster to work properly?

Right. The client does not feel any difference.

If the owner node that owns a queue in the cluster expires and the queue is declared to have a durable attribute, can the queue be successfully redeclared from another node?

No, in this case, you will get a 404 NOT_FOUND error. The node to which the queue belongs can only be used after the queue is restored. However, if the queue itself does not have a durable attribute, it can be redeclared on another node.

What is the impact of the failure of node in cluster on consumer? If a mirrored queue is created in cluster, what will be the impact of node failure on consumer?

If the node to which the consumer is connected fails (whether or not the node is the owner node of the queue to which the consumer is subscribed), the consumer will perform the reconnection logic according to the standard behavior when it finds that the TCP connection is broken, and reconstruct the corresponding fabric according to the "Assume Nothing" principle. If the invalid node is the owner node of the consumer subscription queue, the consumer can only detect the termination of the subscription relationship with the queue through the Consumer Cancellation Notification mechanism, otherwise there will be the problem of waiting without any news.

Can RabbitMQ cluster be used in geographically separate data centers?

I can't. First, you cannot control which node in the cluster the created queue is actually distributed on (this is usually the case when using the HAProxy + cluster model), which may lead to a variety of common problems in cross-regional access; second, Erlang's OTP communication framework has limited tolerance for latency, which may trigger a variety of timeouts, resulting in business processing fatigue Third, the problem of connection failure on the WAN will lead to the classic "brain fissure" problem that RabbitMQ cannot currently deal with (this problem is mainly Mnesia).

These are all the contents of the article "how to achieve Cluster Management in RabbitMQ". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.