Shulou(Shulou.com)05/31 Report--

This article mainly introduces the use of RabbitMQ message queue, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

1. History

RabbitMQ is an open source implementation of AMQP (Advanced Message Queue) developed by erlang. In fact, the emergence of AMQP is to meet the needs of the broad masses of people. Although there are many public standards in the world of synchronous message communication (such as COBAR's IIOP, or SOAP, etc.), this is not the case in asynchronous message processing. Only large enterprises have some commercial implementations (such as Microsoft's MSMQ, IBM's Websphere MQ, etc.). Therefore, in June 2006, Cisco, Redhat, iMatix and so on jointly formulated the open standard of AMQP.

RabbitMQ is developed by RabbitMQ Technologies Ltd and provides commercial support. The company was acquired by SpringSource, a division of VMWare, in April 2010. It was incorporated into Pivotal in May 2013. In fact, VMWare,Pivotal and EMC are essentially the same family. The difference is that VMWare is an independently listed subsidiary, while Pivotal integrates some of the resources of EMC and is not listed now.

two。 Application scenario

Let's get back to the point. What is the problem solved by RabbitMQ, or AMQP, or what is its application scenario?

For a large software system, it will have many components or modules or subsystems or (subsystem or Component or submodule). So how do these modules communicate? This is very different from the traditional IPC. Many of the traditional IPC are on a single system, module coupling is very large, not suitable for expansion (Scalability); if you use socket, then different modules can indeed be deployed to different machines, but there are still many problems to be solved. For example:

1) how do the sender and receiver of the message maintain this connection, and if the connection of one party is broken, how does the data during this period be lost?

2) how to reduce the coupling between sender and receiver?

3) how to make the receiver with high Priority receive the data first?

4) how to achieve load balance? Effectively balance the recipient's load?

5) how to effectively send the data to the relevant recipient? In other words, the receiver will subscribe different data, how to do effective filter.

6) how to be extensible and even send this communication module to cluster?

7) how to ensure that the receiver receives complete and correct data?

The AMDQ protocol solves the above problems, while RabbitMQ implements AMQP.

3. System architecture

It may not be appropriate to be the system architecture, and it may be more appropriate to call the system architecture of the application scenario.

The copyright of this system architecture belongs to sunjun041640.

RabbitMQ Server: also known as broker server, it's not a food truck, it's a delivery service. It's RabbitMQisn't a food truck, it's a delivery service. His role is to maintain a route from Producer to Consumer to ensure that data can be transmitted in a specified way. But this guarantee is not a 100% guarantee, but it is sufficient for ordinary applications. Of course, for commercial systems, you can do another layer of data consistency of guard, you can completely ensure the consistency of the system.

Client A & B: also known as Producer, the sender of the data. Createmessages and publish (send) them to a broker server (RabbitMQ). A Message has two parts: payload (payload) and label (label). As the name implies, payload is the data transferred. Label is the name of exchange, or a tag, which describes payload, and RabbitMQ uses this label to decide which Consumer to send the Message to. AMQP only describes label, and RabbitMQ determines the rules for how to use this label.

Client 1, 2, 3: also known as Consumer, the receiver of the data. Consumersattach to a broker server (RabbitMQ) and subscribe to a queue. Compare queue to a mailbox with a name. When a Message arrives at a mailbox, RabbitMQ sends it to one of its subscribers, Consumer. Of course, it is possible to send the same Message to many Consumer. In this Message, only payload,label has been deleted. For Consumer, it doesn't know who sent this message. Is that the agreement itself does not support it. But of course, if the payload sent by Producer contains information about Producer, it's a different matter.

There are three more concepts that need to be clarified for the correct transfer of data from Producer to Consumer: exchanges and queues and bindings.

Exchanges are where producers publish their messages.

Queuesare where the messages end up and are received by consumers

Bindings are how the messages get routed from the exchange to particular queues.

There are several other concepts that are not indicated in the above figure, that is, Connection (connection), Channel (channel, channel).

Connection: it's just a TCP connection. Both Producer and Consumer are connected to RabbitMQ Server through TCP. As we can see later, the starting point of the program is to establish this TCP connection.

Channels: virtual connection. It is established in the TCP connection above. Data flow takes place in Channel. In other words, generally speaking, the program starts to establish a TCP connection, and the second step is to establish the Channel.

So why use Channel instead of directly using TCP connections?

For OS, it is costly to establish and close TCP connections. Frequent establishment and closure of TCP connections have a great impact on the performance of the system, and the number of TCP connections is also limited, which also limits the ability of the system to deal with high concurrency. However, there is no such cost in establishing a Channel in an TCP connection. For Producer or Consumer, multiple Channel can be used concurrently for Publish or Receive. Some experiments show that 1s data can be Publish20K data packets. Of course, different packet sizes are certainly different for different hardware environments, but I just want to make it clear that this is enough for a normal Consumer or Producer. If it's not enough, you should be thinking about how to refine split your design.

4. Further details clarify 4.1 use ack to confirm the correct delivery of Message

By default, if the Message has been correctly received by a Consumer, then the Message will be removed from the queue. Of course, it is also possible to send the same Message to many Consumer.

If a queue is not subscribed by any Consumer Subscribe, then if the queue has data arriving, the data will be cache and will not be discarded. When there is a Consumer, the data is immediately sent to the Consumer, and when the data is correctly received by the Consumer, the data is deleted from the queue.

So what is the correct receipt? Through ack. Each Message is acknowledged (ack). We can display the ack in the program, or we can automatically ack. If there is data that is not ack, then:

RabbitMQ Server will send this message to the next Consumer.

If the app has bug and forgets ack, RabbitMQ Server will no longer send data to it because Server believes that the Consumer has limited processing power.

And the mechanism of ack can play the role of current limit (Benefitto throttling): send ack after the data is processed by Consumer, or even send ack after extra delay, which will be effective load of balance Consumer.

Of course, for practical examples, for example, we may merge some data, such as data within merge 4s, and then get the data after sleep 4s. Especially in the state of the listening system, we do not want all the state to be delivered in real time, but we want to have a certain delay. This reduces some IO, and the end user will not feel it.

4.2 Reject a message

There are two ways, the first kind of Reject allows RabbitMQ Server to send the Message to the next Consumer. The second is to delete the Message immediately from the queue.

4.3 Creating a queue

Both Consumer and Procuder can create queue through queue.declare. For a Channel, Consumer cannot declare one queue but subscribe to another queue. Of course, you can also create a private queue. This way only app itself can use this queue. Queue can also be deleted automatically, and queue marked as auto-delete will be deleted automatically after the last Consumer unsubscribe. What if you are creating an existing queue? Then it won't have any effect. It is important to note that there is no effect, that is, if the second creation parameter is not the same as the first time, then the operation is successful, but the properties of the queue will not be modified.

So who should be responsible for creating this queue? Is it Consumer or Producer?

If queue doesn't exist, of course Consumer won't get any Message. But if queue does not exist, then Producer Publish's Message will be discarded. So, in order not to lose data, Consumer and Producer are both try to create the queue! In any case, there will be nothing wrong with this interface.

Queue's handling of load balance is perfect. For multiple Consumer, RabbitMQ uses a circular (round-robin) way to send to different Consumer evenly.

4.4 Exchanges

As you can see from the architecture diagram, Procuder Publish's Message has entered Exchange. Then through "routing keys", RabbitMQ will find out which queue the Message should be put in. Queue is also bound through this routing keys.

There are three types of Exchanges:direct, fanout,topic. Each implements a different routing algorithm (routing algorithm).

Direct exchange: if the routing key matches, then the Message is passed to the corresponding queue. In fact, when queue is created, it automatically binds that exchange with the name of queue as routing key.

Fanout exchange: broadcasts to the responding queue.

Topic exchange: perform pattern matching on key, such as queue where ab* can be passed to all ab*.

4.5 Virtual hosts

Each virtual host is essentially a RabbitMQ Server, with its own queue,exchagne, bings rule, and so on. This ensures that you can use RabbitMQ in many different application.

Thank you for reading this article carefully. I hope the article "what is the use of RabbitMQ message queue" shared by the editor will be helpful to everyone? at the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.