Shulou(Shulou.com)06/03 Report--

The following brings you RabbitMQ cluster architecture model and building Rabbitmq-Mirror image cluster, hoping to give you some help in practical application. There are many things involved in load balancing, not many theories, and there are many books on the Internet. Today, we will use the accumulated experience in the industry to do an answer.

RabbitMQ cluster architecture model

For Rabbitmq is a single point of application, if the entire message mq of rabbitmq will be amortized, and all the highly available parts of the message in mq will be particularly important, four cluster architecture solutions are provided in mq:

1. Active / standby mode (Warren)

2. Mirror mode (Mirror)

3. Remote mode (Shovel)

4. Multiple active mode (Federation)

(guess what you like: what is the principle of RabbitMQ clustering)

The most direct mode in our development is the active and standby mode: it mainly implements the high-availability cluster of RabbitMQ. Generally, in the case of concurrency and low amount of data, this model is very easy to use and simple, and the active and standby mode is also called Warren mode.

(guess you like it: the difference between mirroring and clustering in rabbitmq)

That is, one master and one standby, there are at least two CVMs for the cluster, so one of these two servers is working and the other is idle. Note that the master and slave of this server is different from our previous master and slave. The master and slave is one as the master server and the other as the slave server. Although these two servers are data synchronization, the master is responsible for reading and writing, and the slave is only responsible for read-only, while the master and slave is one working and one idle. If a server goes down, then the backup server is switched over. If possible, for load balancers, one is only busy with work and the other is idle, and this kind of production is rarely used. This will cause a waste of our resources.

Mirror mode: the very classic cluster mode is the Mirror mirror mode, which ensures that 100% data is not lost. It is also the most used in practical work, and the implementation of the cluster is also very simple. Generally, large Internet companies will build this kind of mirror cluster mode. The principle is mainly extended on the basis of active and standby. All the node devices in the cluster are synchronized, every queue. The configuration information in the switch is synchronized with our data. For these images to work at the bottom at the same time, use a load balancer, nginx or haproxy, for load balancing.

Remote mode: remote mode can achieve dual-active mode, referred to as Shovel mode, the so-called Shovel is that we can copy messages in different data centers, we can interconnect two MQ clusters across regions, such as a cluster, we will put them in a computer room, so if there are some accidental power outages or natural disasters in the computer room in Beijing, the cluster will be down. Well, in our large-scale applications that require extremely high data, we need to set up a multi-active or dual-active mode, that is, to build multiple data centers or multiple clusters, then these clusters can be placed in Shanghai and one in Beijing. And it should be placed in Guangzhou. The data of the three clusters are all synchronized. If there is a problem with any cluster in the middle, switch flexibly immediately. Well, if these three clusters are accessible, we may choose which cluster or data center to access first according to distance or access speed. All multi-activity modes are generally called disaster recovery mechanism when banks are developing. At least two sets of clusters are built and placed in different regions. If there is a problem, switch immediately so that the whole system will not be down. This is the multi-activity mode. In the multi-active mode, MQ also provides the corresponding implementation method, the early use of the Shovel pattern, this pattern is a mode of mq itself, mainly to distribute messages in different data centers for load distribution, mainly to achieve cross-regional interconnection between two mq clusters.

Well, this shovel model requires a unified version, what level the network achieves, and the configuration is also a bit complicated, which has been eliminated at present, and multi-active mode will be used in the case of real data replication.

Multi-active mode: this mode is also the mainstream mode for remote data replication, which relies on the fedrtation plug-in of rabbimq.

It is not something that comes with mq itself, but is extended on mq, and this extension implements reliable AMQP data communication, so it is relatively easy to configure, which is equivalent to shovel after configuration simplification.

First, build Rabbitmq-Mirror image cluster HA implementation.

Configuration

Mq and erlang version

RabbitMQ is developed using Erlang

Erlang is a general concurrency-oriented programming language. Erlang is a structured, dynamically typed programming language with built-in parallel computing support.

It is much easier to use Erlang to write distributed applications. The Erlang runtime environment is a virtual machine, a bit like a Java virtual machine, so that once the code is compiled, it can also run anywhere.

Download address: https://www.rabbitmq.com/download.html

Download the latest rabbitmq directly to the local

Rabbitmq also needs to rely on the running environment of erlang

Https://www.erlang-solutions.com/resources/download.html

Select centos, find the latest version of 22.2.5 and download it locally.

[root@m2 mq-soft] # lsesl-erlang_22.2.8-1~centos~7_amd64.rpm rabbitmq-server-3.8.3-1.el7.noarch.rpm

Turn off the firewall:

# systemctl stop firewalld#systemctl disable firewalld

Turn off selinux:

# sed-I 'setenforce setenforce' / etc/selinux/config # permanent # temporary

2 Centos environments where Rabbitmq is installed, and the background is used to start when starting.

Start the mq command in the background: rabbitmq-server-detached

Here I have deployed MQ in advance. If you want to know how to deploy MQ, you can read an article.

Modify hostnamectl set-hostname m1/m2

Add vim / etc/hosts to the list of hosts

[root@localhost] # more / etc/hosts192.168.30.27 m1192.168.30.29m2

Mq is very sensitive to hostnames, even if it is not restarted after using hostnamectl set-hostname M1 hostname, it is also a problem of cluster non-communication.

Reboot restart

Copy .erlang.cookie

.erlang.cookie is an erlang distributed token file. All devices in the cluster need to

Holding the same .erlang.cookie file allows you to communicate with each other.

That is to say, you have a student ID card, and it also has a student ID card. You are students, so you can go to school together, that is, a token file.

Find the file at the end of the .cookie and see a string, that is, two hosts need to hold the same cookie file of erlang to communicate with each other.

[root@m1 ~] # find /-name * .cookie / var/lib/rabbitmq/.erlang.cookie [root@m1 ~] # more / var/lib/rabbitmq/.erlang.cookieZCSNJQYQPNPPLIYWOVBJ

Copy, copy the cookie file over

M1 # scp / var/lib/rabbitmq/.erlang.cookie root@192.168.30.24:/var/lib/rabbitmq/root@192.168.30.24's password:

Execute chmod on two computers to authorize 400 pairs of owners to read

M1 chmod 400 / var/lib/rabbitmq/.erlang.cookie m2 chmod 400 / var/lib/rabbitmq/.erlang.cookie

Configure the image cluster

Execute the following command on the m2 server to replicate with the M1 server

-rabbitmqctl stop_app suspension of service

-rabbitmqctl join_cluster rabbit@m1 joins the cluster of M1

-rabbitmqctl start_app starts the service

-rabbitmqctl cluster_status verifies the cluster result

Terminate and start the application

Rabbitmqctl start_app launches the application

Rabbitmqctl stop_app terminates the application

Operate on mq2

[root@m2 ~] # rabbitmqctl stop_appStopping rabbit application on node rabbit@m2... [root@m2 ~] # rabbitmqctl join_cluster rabbit@m1Clustering node rabbit@m2 with rabbit@m1 [root@m2 ~] # rabbitmqctl start_appStarting node rabbit@m2... [root@m2 ~] # rabbitmqctl cluster_statusCluster status of node rabbit@m2... BasicsCluster name: rabbit@m1Disk Nodesrabbit@m1rabbit@m2Running Nodesrabbit@m1rabbit@m2Versionsrabbit@m1: RabbitMQ 3.8.3 on Erlang 22.2.8rabbit@m2: RabbitMQ 3.8.3 on Erlang 22.2.8

Check that if you are already registered, if you are in general production, either disable this guest user or change a more complex password with it.

Creating test to view m2 will also be synchronized in.

Generally speaking, we need to connect an IP address in the program, so this place needs such a load balancer to implement. Here we use HAproxy to implement it.

IV. Problems encountered in mirror mode

When equipped with two mq, for the client, that is, java, when we go to connect, we need a unified entrance to make direct calls to the system or messages, that is, which server we need to connect to for data processing. If this is the case, when M1 hangs up, we need to manually change the address of the Java client. Another problem is that M1 works hard and m2 has nothing to do. And the use of our resources is also unreasonable, then the load balancer came into being.

When there is haproxy, it will forward the request to M1 or m2 for division of labor, such as installing a simple algorithm polling mode and sharing the tasks equally, so that the resources can be used reasonably. For Java clients, the address of proxy can be configured directly, while haproxy can know which servers can send messages through heartbeat, such as when a m2 machine is down. It will automatically withdraw our service and use other nodes to provide services.

HAProxy proxy server

HAProxy is a technology that provides high availability, load balancing and TCP-based (layer 4)

And HTTP (layer 7) application agent software, which supports virtual hosts, it is free,

A fast and reliable solution.

In RabbitMQ cluster image mode, HAProxy is used as Tcp to provide node load.

LB-LoadBalance and fault detection.

Install haproxy

Yum-y install haproxy

Address of the configuration file

[root@haproxy ~] # find /-name haproxy.cfg/etc/haproxy/haproxy.cfg

Compile configuration file

#-# common defaults that all the 'listen' and' backend' sections will#use if not designated in their block#-- -I don't want the one below. Then configure the following # to listen to the MQ cluster

I put the listening HAproxy configuration address of the MQ cluster in the warehouse, and send me the public key if necessary.

Git clone git@gitee.com:zhaocheng172/rabbitmq-haproxy.git

Effective profile

[root@haproxy] # haproxy-f / etc/haproxy/haproxy.cfg

The connection haproxy access address is the connection password rabbitmq set above. Here is the login password and url address in the configuration file.

If you log in, you can see that our MQ has been monitored. If you use JAVA to send the message of the system call, then the message will be provided and consumed.

After reading the above about RabbitMQ cluster architecture model and building Rabbitmq-Mirror image cluster, if there is anything else you need to know, you can find out what you are interested in in the industry information or find our professional and technical engineer to answer, the technical engineer has more than ten years of experience in the industry.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.