Shulou(Shulou.com)06/03 Report--

This blog briefly introduces the configuration and installation of rabbitmq single node and cluster, the use of rabbitmq commands, and the graphical interface.

Without saying much, let's get to the point. First, let's introduce rabbitmq:

History of rabbitmq:

RabbitMQ is an open source implementation of AMQP (Advanced Message Queue) developed by erlang. In fact, the emergence of AMQP is to meet the needs of the broad masses of people. Although there are many public standards in the world of synchronous message communication (such as COBAR's IIOP, or SOAP, etc.), this is not the case in asynchronous message processing. Only large enterprises have some commercial implementations (such as Microsoft's MSMQ, IBM's Websphere MQ, etc.). Therefore, in June 2006, Cisco, Redhat, iMatix and so on jointly formulated the open standard of AMQP.

RabbitMQ is developed by RabbitMQ Technologies Ltd and provides commercial support. The company was acquired by SpringSource, a division of VMWare, in April 2010. It was incorporated into Pivotal in May 2013. In fact, VMWare,Pivotal and EMC are essentially the same family. The difference is that VMWare is an independently listed subsidiary, while Pivotal integrates some of the resources of EMC and is not listed now.

RabbitMQ's official website is http://www.rabbitmq.com.

Rabbitmq is an open source implementation of Advanced Message Queuing Protocol (AMQP) provided by LShift and written by Erlang, which is famous for its high performance, robustness, and scalability, so it inherits these advantages.

Two major components of Rabbitmq:

Exchange and Queue (there will be changes in AMQP 1.0). As shown in the following figure, the green X is Exchange and the red one is Queue, both of which are on the Server side, also known as Broker. This part is implemented by RabbitMQ, while the blue one is on the client side. There are usually two types: Producer and Consumer:

Rabbitmq supports two protocols:

AMQP protocol: default MQTT protocol: it needs to be enabled manually (how to enable it is described below)

What is AMQP:

The advanced message queuing protocol makes it possible for full-function interoperability between client applications and message middleware servers that comply with the specification.

AMQP range:

In order to fully realize the interoperability of message middleware, it is necessary to fully define the functional semantics of network protocols and message broker services.

Therefore, AMQP defines network protocols and proxy services as follows:

A set of defined message exchange functions, that is, the "advanced message exchange protocol model". The AMQP model includes a set of functional modules for routing and storing messages, as well as a set of rules for exchanging messages between these modules.

A network wire-level protocol (data transfer format) through which client applications can interact with the message broker and its implemented AMQP model.

It is possible to implement only some of the semantics in the AMQP protocol specification, but we believe that a clear description of these semantics will help to understand the protocol

What is the MQTT protocol:

MQTT (Message Queuing Telemetry Transport, message queuing Telemetry Transmission) is an instant messaging protocol developed by IBM, which may become an important part of the Internet of things. The protocol supports all platforms, can connect almost all connected objects to the outside, and is used as a communication protocol for sensors and brakes (such as connecting houses through Twitter).

MQTT protocol features:

A protocol designed for the communication of a large number of remote sensors and control devices with limited computing power and operating in low-bandwidth, unreliable networks, it has the following main characteristics:

1. Use the publish / subscribe message model to provide one-to-many message publishing and uncouple the application.

2. Message transmission that shields the load content

3. Use TCP/IP to provide network connection

4. There are three kinds of message publishing service quality:

"at most once," the release of messages depends entirely on the underlying TCP/IP network. Messages can be lost or duplicated. This level can be used in the case of environmental sensor data, it doesn't matter if you lose a read record, because there will be a second transmission soon. "at least once" to ensure that the message arrives, but message repetition may occur. "only once", make sure the message arrives once. This level can be used in situations where duplicate or lost messages can lead to incorrect results in a billing system.

5. Small transmission with low overhead (fixed length header is 2 bytes). Protocol switching is minimized to reduce network traffic.

6. The mechanism for notifying the parties concerned of abnormal client interruptions using Last Will and Testament features

Note: when using rabbitmq, port 5672, namely AMQP protocol, is enabled by default, but some Android cannot use AMQP protocol, so we need to enable MQTT protocol.

Usage scenarios of RabbitMQ:

What is the problem solved by RabbitMQ, or AMQP, or what is its application scenario?

For a large software system, it will have many components or modules or subsystems or (subsystem or Component or submodule). So how do these modules communicate? This is very different from the traditional IPC. Many of the traditional IPC are on a single system, module coupling is very large, not suitable for expansion (Scalability); if you use socket, then different modules can indeed be deployed to different machines, but there are still many problems to be solved. For example:

1) how do the sender and receiver of the message maintain this connection, and if the connection of one party is broken, how does the data during this period be lost?

2) how to reduce the coupling between sender and receiver?

3) how to make the receiver with high Priority receive the data first?

4) how to achieve load balance? Effectively balance the recipient's load?

5) how to effectively send the data to the relevant recipient? In other words, the receiver will subscribe different data, how to do effective filter.

6) how to be extensible and even send this communication module to cluster?

7) how to ensure that the receiver receives complete and correct data?

The AMDQ protocol solves the above problems, while RabbitMQ implements AMQP.

System framework:

I will not introduce the working principle of rabbitmq here. If you want to know more, you can take a look at the official website. Here, we will provide you with the system framework diagram:

RabbitMQ Server: also known as broker server, it's not a food truck, it's a delivery service. It's RabbitMQ isn't a food truck, it's a delivery service. His role is to maintain a route from Producer to Consumer to ensure that data can be transmitted in a specified way. But this guarantee is not a 100% guarantee, but it is sufficient for ordinary applications. Of course, for commercial systems, you can do another layer of data consistency guard (protection, protective devices), you can completely ensure the consistency of the system.

Client A & B: also known as Producer, the sender of the data. Create messages and publish (send) them to a broker server (RabbitMQ). A Message has two parts: payload (payload) and label (label). As the name implies, payload is the data transferred. Label is the name of exchange, or a tag, which describes payload, and RabbitMQ uses this label to decide which Consumer to send the Message to. AMQP only describes label, and RabbitMQ determines the rules for how to use this label.

Client 1, 2, 3: also known as Consumer, the receiver of the data. Consumers attach to a broker server (RabbitMQ) and subscribe (subscription) to a queue. Compare queue to a mailbox with a name. When a Message arrives at a mailbox, RabbitMQ sends it to one of its subscribers, Consumer. Of course, it is possible to send the same Message to many Consumer. In this Message, only payload,label has been deleted. For Consumer, it doesn't know who sent this message. Is that the agreement itself does not support it. But of course, if the payload sent by Producer contains information about Producer, it's a different matter.

Connection: it's just a TCP connection. Both Producer and Consumer are connected to RabbitMQ Server through TCP. As we can see later, the starting point of the program is to establish this TCP connection.

Channels: virtual connection. It is established in the TCP connection above. Data flow takes place in Channel. That is to say, generally speaking, the program starts to establish a TCP connection, and the second step is to establish the Channel.

After introducing the nonsense of rabbitmq, let's go straight to the topic of deploying rabbitmq clusters:

What is installed in this demonstration is installed by yum, and it is also recommended to set up a yum installation, otherwise the version of erlang environment and rabbtmq may have problems and deployment problems may occur.

Install the basic environment:

Yum-y install wget vim bash-completion lrzsz nmap telnet tree net-tools bind-utils lsof ntpdate iotop

Prepare the erlang environment:

Wget-O/etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

Install rabbitmq:

Yum-y install erlangyum-y install rabbitmq-server

Erlang version: erlang.x86_64 0:R16B-03.18.el7

Rabbitmq version: rabbitmq-server.noarch 0vl 3.3.5-34.el7

Check the installation results:

Rpm-qa | grep rabbitmq

Start and view status:

Systemctl status rabbitmq-server # View rabbitmq transition systemctl start rabbitmq-server # start rabbitmqsystemctl stop rabbitmq-server # turn off rabbitmq

Rabbitmq in many cases, the use of systemctl restart rabbitmq-server is ineffective, you must stop after the start

Open the Web management page:

Rabbitmq-plugins enable rabbitmq_management

Enable MQTT protocol:

Rabbitmq-plugins enable rabbitmq_mqttAMQP protocol port default: 5672Web page default port: 15672MQTT protocol default port: 1883 default login user name (admin permission): guest default login password: guest

If you want to install a single node, this is the end of it. If you want to install a cluster, please keep going.

First of all, let's talk about the deployment of rabbitmq clusters on a single machine with multiple nodes:

First, follow the steps above to install a rabbitmq, and then do the following:

Make sure the rabbitmq node is turned off: rabbitmqctl stop

Method 1:

Set the environment variable to specify the port and node name:

RABBITMQ_NODE_PORT=5672 RABBITMQ_NODENAME=rabbit

Start the node in the background:

Rabbitmq-server-detached

Repeat the above, but the port and rabbitmq node names cannot be the same:

RABBITMQ_NODE_PORT=5673 RABBITMQ_NODENAME=rabbit_01rabbitmq-server-detachedRABBITMQ_NODE_PORT=5674 RABBITMQ_NODENAME=rabbit_02rabbitmq-server-detached

Method 2:

Start by writing a master configuration file: (rabbitmq does not have a master configuration file by default)

Vim / etc/rabbitmq/rabbitmq-env.confNODE_PORT=5672NODENAME=rabbitNODE_PORT=5673NODENAME=rabbit_01NODE_PORT=5674NODENAME=rabbit_03systemctl start rabbitmq-server

No matter which method is used before, the way to join the cluster is the same:

Listen to the application on the node first:

# stop rabbit_01 Node apply rabbitmqctl-n@localhost rabbit_01 stop_app# add rabbit_01 to rabbit Node rabbitmqctl-n@localhost rabbit_01 join_cluster rabbit@localhost# stop rabbit_02 Node apply rabbitmqctl-n@localhost rabbit_02 stop_app# add rabbit_02 to rabbit Node rabbitmqctl-n@localhost rabbit_02 join_cluster rabbit@localhost

View node information:

# Note: only two messages in the cluster are shown here. If you add them according to the above operation, three node information Cluster status of node rabbit@localhost. [{nodes, [{disc, [rabbit@localhost,rabbit_01@localhost]}]}, {running_nodes, [rabbit@localhost]}, {cluster_name,}, {partitions, []}, {alarms, [{rabbit@localhost, []}]}]

Here is an introduction to multi-node deployment:

* Note: when multiple nodes are different from single nodes, in a multi-machine environment, if you want to deploy multiple nodes in a cluster cluster, you need to pay attention to two aspects:

To ensure that the nodes that need to be deployed need to have the same Erlang Cookie in the same local area network, otherwise they cannot communicate. In order to ensure the complete consistency of cookie, we adopt the method of copy from one node.

Main topic:

IP hostname rabbitmq node name 192.168.3.10node1rabbitmq1192.168.3.100node3rabbitmq2192.168.3.205node4rabbitmq3

It's all a little awkward. The hosts file in the mainframe has already been matched, so it's hard to change it. Just use it, but when you look at it, you should pay attention to it. I don't have node2 in it.

Turn off the firewall, or configure the firewall rules, otherwise there will be problems in joining the cluster later.

Configure the node1 host hosts file: (make sure the hostname can be resolved)

Vim / etc/hosts192.168.3.10 node1192.168.3.100 node3192.168.3.205 node4scp / etc/hosts root@192.168.3.100:/etc/hostsscp / etc/hosts root@192.168.3.205:/etc/hosts

Install rabbitmq (the same operation on the three sets):

Yum-y install wget vim bash-completion lrzsz nmap telnet tree net-tools bind-utils lsof ntpdate iotopwget-O/etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repoyum-y install erlang rabbitmq-server

Configure the same Erlang Cookie:

Make sure that the cookie of the three is the same, so that the erlang is in an environment.

Start rabbitmq first, otherwise there is no cookie, and cookie is a hidden file:

Note: all three orders need to be started by rabbitmq, and then closed.

Execute on the node1 node:

Scp / var/lib/rabbitmq/.erlang.cookie root@node3:/var/lib/rabbitmqscp / var/lib/rabbitmq/.erlang.cookie root@node4:/var/lib/rabbitmq

Restart the service to see if there is an error:

* restart all three nodes to see if there are any errors * systemctl stop rabbitmq-serversystemctl start rabbitmq-server

Note: if there is an error, it must be a .erlang.cookie problem, either delete the .erlang.cookie and regenerate it, or see if you forgot to turn off the service and configure cookie. Be sure to pay attention to these details.

Join the node in the cluster # ensure that the service is started and operate on the node3 node # add node3 to the node1 node cluster: rabbitmqctl stop_app # stop the node service rabbitmqctl reset # reset the node configuration rabbitmqctl join_cluster rabbit@node1 # add the rabbitmq of the node3 node to the node1 # rabbitmqctl join_cluster rabbit@ is written in a fixed format rabbitmqctl start_app # start the node

Let's go on to talk about node4 joining the node1 cluster:

7. View the status of the cluster:

Rabbitmqctl cluster_status

Note: we can see that these points are all disc (disk nodes). We can manually adjust the cluster node mode to memory nodes. Of course, three disk nodes are the safest, but it is a waste of resources. Normally, we can adjust it to two disks and one memory.

The following is how to set the node mode of the cluster:

Ram: memory

Disc: disk

The default node to join the cluster is a disc node. There are two ways to set a ram node:

Method 1:

Set type when joining the cluster:

Rabbitmqctl stop_apprabbitmqctl join_cluster-ram rabbit@node1rabbitmqctl start_app

Method 2:

After joining the cluster, change the disc node to the ram node:

Rabbitmqctl stop_apprabbitmqctl change_cluster_node_type ramrabbitmqctl start_app

I used the second method to change node4 from a disk node to a memory node:

It is sometimes difficult for us to operate on the command line, so we turn on Web dot-by-dot mode: (three sets of the same operation)

Rabbitmq-plugins enable rabbitmq_managementsystemctl stop rabbitmq-serversystemctl start rabbitmq-server

To see if the port is up, the default port is 15672:

# three sets of the same operation

Manually enable MQTT protocol (port 1883). Only AMQP protocol (port 5672) is enabled by default.

Rabbitmq-plugins enable rabbitmq_mqttsystemctl stop rabbitmq-serversystemctl start rabbitmq-server

View port status:

Cluster creation user

# as long as it is created in one node, all the other former nodes in a cluster will have this user

Rabbitmqctl add_user hcb hcb@pwd # create user password rabbitmqctl set_permissions hcb ". *" # empower hcb rabbitmqctl set_user_tags hcb administrator # give hcb adminsystemctl restart rabbitmq-server # restart the service

At this point, we can log in with this user password in the web page.

Http://192.168.3.10:15672

# it is OK to log in with the default user guest, and the default is admin permission

After logging in, we can see all the information about the cluster:

I won't say much about how to use web here, it's all very simple, if you can't, you can contact me, on-site distance teaching.

RabbitMQ load balancing

Choose the open source HAproxy to load the rabbitmq cluster

Install epel:

Wget-O / etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

Install HA:

Yum-y install haproxy

Configure HA:

Cp / etc/haproxy/haproxy.cfg / etc/haproxy/haproxy.cfg.bakvim / etc/haproxy/haproxy.cfg

First, let's talk about the meaning of the configuration file of haproxy:

Global # global settings log 127.0.0.1 local0 # log output configuration, all logs are recorded locally Output from local0 # log loghost local0 info maxconn 4096 # maximum number of connections chroot / usr/local/haproxy uid 99 # running user uid gid 99 # running user group daemon # running haproxy nbproc 2 later # start 2 haproxy instances pidfile / usr/local/haproxy/haproxy.pid # write all processes to the pid file # debug # quietdefaults # default setting # log global log 127.0.0.1 local3 # output orientation of log files mode http # categories handled Http mode is used by default Can be configured as tcp for layer 4 message forwarding option httplog # log category, using httplog option dontlognull option forwardfor # if the back-end server needs to obtain the parameters that the client needs to configure for real ip You can obtain the client ip option httpclose # from Http Header to actively close the http channel after each request. Haproxy does not support keep-alive, and can only simulate the implementation of this mode. If retries 3 # 3 connection fails, the server is considered unavailable, mainly through the following check check option redispatch # when the corresponding server of serverid hangs up. Force direct to other health server maxconn 2000 # maximum number of connections contimeout 5000 # connection timeout clitimeout 50000 # client connection timeout srvtimeout 50000 # server connection timeout Time stats auth Frank:Frank # sets the user and password of the monitoring page: Frankstats hide-version # hides the HAproxy version information of the statistics page listen rabbitmq_local_cluster VIP:5670 / / the previous paragraph IP For product and consumer to choose, since port 5672 is already used by default, select port 5670 mode tcp / / load balancing option balance roundrobin / / polling algorithm to send the load to the background server server rabbit 192.168.3.10 server rabbit 192.168.3.10 mode tcp 5000 rise 5000 fall 3ram / cluster node configuration in load balancing The rabbit node selected here is server rabbit 192.168.3.100 check inter 5000 rise 2 fall 3 server rabbit 192.168.3.205 check inter 5000 rise 2 fall 3 listen private_monitoring: 8100 mode http option httplog stats enable stats uri / stats stats refresh 60s

# actual configuration, such as:

Global log 127.0.0.1 local0 info maxconn 4096 stats socket / tmp/haproxy.socket uid haproxy mode 770 level admin daemondefaults log global mode tcp option tcplog option dontlognull retries 3 option redispatch maxconn 2000 timeout connect 5s timeout client 120s timeout server 120slisten rabbitmq_local_cluster 0.0.0.0:5670 mode tcp balance roundrobin server cacheclod1 192.168.3.10:5672 check inter 5000 rise 2 fall 3 server cacheclod2 192.168.3.100:5672 check inter 5000 Rise 2 fall 3 server cacheclod3 192.168.3.205:5672 check inter 5000 rise 2 fall 3listen private_monitoring: 8100 mode http option httplog stats enable stats uri / stats stats refresh 5s

Start HAproxy:

Systemctl start haproxy

View the haproxy control interface:

Http://192.168.3.10:8100/stats

Where there is a will, there is a way, burning down the bridges, the hundred and two Qin Guan will eventually belong to Chu.

If you live up to your painstaking efforts, you can swallow Wu with three thousand more.

At this point, after the introduction of rabbitmq, there are still a lot of things left to say, such as the two modes of the cluster (mirror mode and normal mode), the configuration file of rabbitmq, and the routing mechanism, which will be explained one by one in later documents.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.