Shulou(Shulou.com)05/31 Report--

What this article shares with you is about the principle of RabbitMQ and what the use scene is. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

one。 Introduction to RabbitMQ

MQ is called Message Queue, and message queuing (MQ) is a method of application-to-application communication. Applications communicate by reading and writing messages in and out of the queue (for the application's data) without the need for dedicated connections to link them. Messaging refers to the communication between programs by sending data in messages, rather than communicating with each other through direct calls, which are usually used for technologies such as remote procedure calls. Queuing refers to applications communicating through queues. The use of queues removes the requirement that the receiving and sending applications execute simultaneously.

RabbitMQ is an open source message queuing system developed in Erlang language and implemented based on AMQP protocol. The main features of AMQP are message-oriented, queue-oriented, routing (including peer-to-peer and publish / subscribe), reliability and security. AMQP protocol is more used in enterprise systems, requiring high data consistency, stability and reliability, followed by performance and throughput requirements.

two。 RabbitMQ usage scenario

1. Decoupling (providing basic ultimate consistency implementation for service-oriented architecture (SOA))

Scenario description: after the user places an order, the order system needs to notify the inventory system. Traditionally, the order system calls the interface of the inventory system.

Disadvantages of the traditional model:

If the inventory system is not accessible, the order reduction will fail, resulting in order failure

Coupling of order system and inventory system

Introduce message queue

Order system: after the user places an order, the order system completes the persistence processing, writes the message to the message queue, and returns the user's order to be issued successfully.

Inventory system: subscribe to the order message and use the pull / push method to obtain the order information. The inventory system carries out inventory operation according to the order information.

If: the inventory system does not work properly when placing an order. It also does not affect the normal issuance of the order, because after the order is issued, the order system writes to the message queue and no longer cares about other subsequent operations. Realize the application decoupling of order system and inventory system

In order to ensure that there must be inventory, you can set the queue size to the inventory quantity, or use other ways to solve the problem.

The message-based model is concerned with "notification", not "processing".

SMS, email notification, cache refresh and other operations are notified using message queues.

The difference and comparison between message queuing and RPC:

RPC: asynchronous call, get the call result in time, have strong consistency result, care about the business call processing result.

Message queue: two asynchronous RPC calls, the contents of which are dumped in the queue and delivered at the right time (off-peak flow control)

two。 Improve efficiency asynchronously

Scenario description: after registration, users need to send registration email and registration SMS. There are two traditional practices. Serial mode; 2. Parallel mode

Extend:

Asynchronous and concurrent weapon: a practice of using CompletionService to improve system performance in real projects

(1) Serial mode: after successfully writing the registration information to the database, send the registration email, and then send the registration text message. When all the above three tasks are completed, return to the client

(2) parallel mode: after the registration information is successfully written into the database, the registration email is sent and the registration SMS is sent at the same time. After the above three tasks are completed, return to the client. The difference with serial is that parallel approach can improve processing time.

The introduction of message queuing will not be a necessary business logic for asynchronous processing. The modified structure is as follows:

3. Flow peaking

Traffic peak shedding is also a common scenario in message queues, which is widely used in second kill or group robbery activities.

Application scenario: system A has 100 requests per second at other times, and the system can run stably. The system has flash sale activity at 8 o'clock every evening, and the number of concurrent requests per second increases to 10, 000, but the maximum processing capacity of the system can only handle 1000 requests per second, so the system crashes and the server crashes.

Previous architecture: a large number of users (1 million users) participated in flash sale activity at the same time through the browser during the peak period at 8 p.m. A large number of requests pour into our system, with a peak of 5000 requests per second, a large number of requests are called to MySQL, and 3000 SQL are expected to be executed per second. But the average MySQL is good enough to handle 2000 requests per second. If 3000 requests are reached, the MySQL may be paralyzed directly and the system cannot be used. But after the peak, it becomes a trough, perhaps only 10,000 users access the system, the number of requests per second is only about 50, the whole system has almost no pressure.

Introduction of MQ: 1 million users have about 5000 requests per second during the peak period. If these 5000 requests are written into MQ, system A can only handle a maximum of 2000 requests per second, because MySQL can only handle 2000 requests per second. System A slowly pulls requests from MQ, pulling 2000 requests per second, no more than the number of requests it can handle per second. MQ, with 5000 requests coming in per second, results in only 2000 requests going out, so there may be hundreds of thousands or millions of requests backlog in MQ during the second kill period (nearly an hour).

About flow peak cutting: what should be done to cut the flow peak of the second kill system?

This short peak backlog is fine, because after the peak, only 50 requests per second enter MQ, but the system is still processing at the rate of 2000 requests per second, so as soon as the peak is over, the system will quickly consume the backlog of messages. Let's calculate that if there is a backlog of 3000 messages in MQ per second, there will be a backlog of 180000 messages in one minute and 10 million messages in an hour. After the peak, the backlog of 10 million messages can be consumed in more than one hour.

three。 Advantages and disadvantages of introducing message queues

Advantages

The advantage is that the above scenario applications have their corresponding advantages in special scenarios, such as decoupling, asynchronism and peak clipping.

Shortcoming

Reduced availability of the system

The more external dependencies introduced into the system, the easier it is for the system to hang up. Originally, it is only good for system A to call the three system interfaces of BCD, and the whole system will operate normally if the four systems of ABCD do not report errors. After the introduction of MQ, although the ABCD system did not go wrong, but after the MQ failed, the whole system will crash.

Increased complexity of the system

With the introduction of MQ, there are more questions to consider. How to ensure that messages are not consumed repeatedly? How to ensure that the message is not lost? How to ensure the order of message delivery?

Consistency problem

A system returns successfully after sending the message, but if any system in the BCD system fails to write the database, it will cause the problem of data inconsistency.

Message queue is a very complex architecture, and there are many advantages to introduce it, but we have to do a variety of additional technical solutions and architectures to avoid the disadvantages it brings. The complexity of introducing MQ system is increased by an order of magnitude, but in some scenarios, even if it is ten times more complex, it is still necessary to use MQ.

The above is the principle of RabbitMQ and what the usage scenario is. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.