Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what are the test questions that RabbitMQ often meets". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. What are the benefits of using RabbitMQ?

1. Decoupling, system A directly calls the code of system B and system C in the code. If system D is connected in the future, system A still needs to modify the code, which is too troublesome!

two。 Asynchronism, writing messages to the message queue, and non-essential business logic running asynchronously to speed up the response

3. Peak cutting, when the amount of concurrency is large, all requests are directly directed to the database, resulting in abnormal database connection

2. What is broker in RabbitMQ? What does cluster mean?

A broker is a logical grouping of one or more erlang node, and a RabbitMQ application is running on the node. Cluster adds the constraint of sharing metadata between node on the basis of broker.

3. Are channel, exchange and queue in the concept of RabbitMQ logical concepts or corresponding to process entities? What is the role of each?

Queue has its own erlang process; exchange is implemented internally as a lookup table that holds binding relationships; and channel is the entity that actually carries out routing work, that is, it is responsible for delivering message to queue according to routing_key. According to the description of the AMQP protocol, channel is a virtual connection over a real TCP connection, all AMQP commands are sent through channel, and each channel has a unique ID. A channel can only be used by a single operating system thread, so the message delivered to a particular channel is sequential. However, multiple channel are allowed on one operating system thread.

4. What is vhost? What does it do?

Vhost can be understood as virtual broker, that is, mini-RabbitMQ server. It contains independent queue, exchange and binding, etc., but the most important thing is that it has an independent authority system, which can be controlled by vhost users. Of course, from a global point of view of RabbitMQ, vhost can be used as a means of isolating different permissions (a typical example is that different applications can run in different vhost).

5. What is the basis for message transmission?

Because the creation and destruction of TCP connections are expensive, and the number of concurrency is limited by system resources, it will cause performance bottleneck. RabbitMQ uses channels to transmit data. Channels are virtual connections established within real TCP connections, and there is no limit to the number of channels on each TCP connection.

6. How is the message distributed?

If the queue has at least one consumer subscription, the message will be sent to the consumer in a round-robin. Each message is distributed to only one subscribed consumer (as long as the consumer can process the message normally and confirm it).

7. How to route the message?

Conceptually, message routing must have three parts: switch, routing, and binding. The producer publishes the message to the switch; the binding determines how the message is routed from the router to a specific queue; the message eventually arrives in the queue and is received by the consumer.

When a message is published to the switch, the message will have a routing key (routing key), which is set when the message is created.

Through the queue routing key, you can bind the queue to the switch.

After the message arrives at the switch, RabbitMQ matches the routing key of the message with the routing key of the queue (there are different routing rules for different switches). If it can be matched to the queue, the message will be delivered to the corresponding queue; if it cannot be matched to any queue, the message will enter the "black hole".

The commonly used switches are divided into the following three types:

Direct: if the routing keys match exactly, the message is delivered to the appropriate queue

Fanout: if the switch receives a message, it will be broadcast to all bound queues

Topic: messages from different sources can reach the same queue. When using topic switches, you can use wildcards, such as "*" to match any text in a specific location, "." Divide the routing key into several parts, "#" matches all rules, etc. Special note: messages sent to the topic switch can not be arbitrarily set the routing_key, must be by the "." Consists of a separated series of identifiers.

8. What is metadata? What are the types of metadata? What does it include? What is the metadata related to cluster? How is metadata saved? How is metadata distributed in cluster?

In non-cluster mode, metadata is mainly divided into Queue metadata (queue names and attributes, etc.), Exchange metadata (exchange names, types and attributes, etc.), Binding metadata (lookup tables that store routing relationships), and Vhost metadata (namespace constraints and security attribute settings for the first three in vhost). In cluster mode, node location information and node relationship information in cluster are also included. According to the type of erlang node, the metadata determines whether it is saved only in RAM or on both RAM and disk. Metadata is fully node distributed in cluster.

The following figure shows the distribution of queue's metadata in both single node and cluster modes.

9. What is the difference between declaring queue, exchange, and performing binding in a single node system and a multi-node cluster system?

A: when you declare queue on a single node, you will get a Queue.Declare-ok response as long as the relevant metadata on the node is changed, while declaring queue on cluster requires that all node on the cluster must be successfully updated with metadata before you can get a Queue.Declare-ok response. In addition, if the node type is RAM node, the changed data is saved only in memory, and if the type is disk node, the data saved on disk is also changed.

Dead-letter queue-dead-letter switch: DLX full name (Dead-Letter-Exchange), called dead-letter switch. When a message becomes a dead-letter, if the queue in which the message resides has a x-dead-letter-exchange parameter, it will be sent to the switch corresponding to the x-dead-letter-exchange value. This switch is called dead-letter switch, and the queue bound to this dead-letter switch is the dead-letter queue.

10. How do I ensure that messages are sent to RabbitMQ correctly?

RabbitMQ uses sender acknowledgement mode to ensure that messages are sent correctly to RabbitMQ. Sender acknowledgement mode: if the channel is set to confirm mode (sender acknowledgement mode), all messages posted on the channel will be assigned a unique ID. Once the message is delivered to the destination queue, or after the message is written to disk (a persistent message), the channel sends an acknowledgement to the producer (including the message's unique ID). If an internal error occurs in RabbitMQ and the message is lost, a nack (not acknowledged) message is sent. The sender acknowledgement mode is asynchronous, and the producer application can continue to send messages while waiting for confirmation. When the confirmation message arrives at the producer application, the callback method of the producer application is triggered to process the confirmation message.

11. How to ensure that the message receiver consumes the message?

Receiver message acknowledgement mechanism: consumers must acknowledge each message after receiving it (message reception and message acknowledgement are two different operations). Only when the consumer acknowledges the message can RabbitMQ safely remove the message from the queue. The timeout mechanism is not used here, and RabbitMQ simply confirms whether the message needs to be resent by breaking the connection to the Consumer. That is, as long as the connection is not broken, RabbitMQ gives Consumer enough time to process the message.

Here are a few special cases:

If the consumer receives the message, disconnects or unsubscribes before confirmation, RabbitMQ assumes that the message has not been distributed and then redistributes it to the consumer of the next subscription. (there may be a hidden danger of repeated consumption of messages, which needs to be removed according to bizId)

If the consumer receives a message but does not acknowledge the message and the connection is not disconnected, RabbitMQ believes that the consumer is busy and will not distribute more messages to the consumer.

12. How to avoid repeated delivery or consumption of messages?

During message production, MQ generates an inner-msg-id for messages sent by each producer as a basis for de-duplication and idempotence (message delivery fails and is retransmitted) to prevent duplicate messages from entering the queue In message consumption, it is required that there must be a bizId in the message body (globally unique for the same business, such as payment ID, order ID, post ID, etc.) as a basis for de-duplication and idempotence, so as to avoid repeated consumption of the same message.

This question is based on the business scenario to answer the following points:

1. For example, you get this message to do insert operation of the database. That's easy. Make a unique primary key for this message, and even if there is repeated consumption, it will lead to primary key conflicts and avoid dirty data in the database.

two。 For example, if you get this message to do the set operation of redis, it will be easy and need not be solved, because no matter how many times you set, the result is the same, set operation is idempotent.

3. If the above two situations don't work, make a big move. Prepare a third-party media to record consumption. Take redis as an example, a global id is assigned to the message, and as long as the message has been consumed, it will be written to redis in the form of Kmurv. Before consumers start spending, they can go to redis to check whether there is a record of consumption.

13. How to solve the problem of data loss?

1. Producer loses data

What if the producer's message is not delivered to the MQ? From the point of view of producers losing data, RabbitMQ provides transaction and confirm patterns to ensure that producers do not lose messages.

The transaction mechanism means that before sending a message, open the transaction (channel.txSelect ()), and then send the message. If there is any exception during the sending process, the transaction will be rolled back (channel.txRollback ()), and if it is successfully sent, the transaction will be submitted (channel.txCommit ()).

The disadvantage, however, is the decline in throughput. Therefore, according to the experience of bloggers, most of them use confirm mode in production. Once channel enters confirm mode, all messages posted on that channel will be assigned a unique ID (starting from 1). Once the message is delivered to all matching queues, rabbitMQ will send an Ack to the producer (the only ID containing the message), which allows the producer to know that the message has reached the destination queue correctly. If rabiitMQ fails to process the message, a Nack message will be sent to you and you can retry the operation.

two。 Message queuing lost data

To deal with the loss of data in message queues, it is generally necessary to turn on the configuration of persistent disks. This persistence configuration can be used in conjunction with the confirm mechanism, and you can send an Ack signal to the producer after the message is persisted to the disk. In this way, if the rabbitMQ dies before the message is persisted to the disk, the producer does not receive the Ack signal, and the producer will automatically resend it.

So how to persist? by the way, it's actually very easy, just the next two steps.

①, set the persistence identifier durable of queue to true, which means that it is a persistent queue.

②, deliveryMode=2 when sending messages

After this setting, even if the rabbitMQ is dead, the data can be restored after reboot. The service may be dead before the message is persisted to the hard disk, which can be done by introducing mirrored-queue, that is, the mirror queue, but there is no guarantee that the message will not be lost (the whole cluster is down).

3. Consumers lose data

Enabling manual confirmation mode can solve this problem.

In ① automatic confirmation mode, consumers hang up and wait for ack messages to return to the queue. The consumer throws an exception and the message is constantly retransmitted until it is successfully processed. The message is not lost, and even if the service dies, the message that is not processed will be returned to the queue, but the exception will cause the message to keep retrying.

② manual confirmation mode. If the consumer dies before it has time to handle it, it will repeatedly send a message to other consumers if it does not respond to the ack. If the listener handles an exception and does not catch the exception, it will repeatedly receive the message and then throw the exception all the time. If the exception is caught but not ack in the finally, the message will be sent repeatedly (retry mechanism).

③ does not confirm the mode, acknowledge= "none" does not use the confirmation mechanism, as soon as the message is sent, it will be removed in the queue immediately, regardless of the client exception or disconnection, it will be removed as soon as it is sent, and will not be retransmitted.

14. The use of dead letter queues and delay queues

Dead letter message:

Message is rejected (Basic.Reject or Basic.Nack) and the value of the requeue parameter is set to false

The message has expired.

The queue reaches the maximum length

Expired message:

There are two ways to set the expiration time of messages in rabbitmq, the first is by setting the queue, after this setting, all messages in the queue have the same expiration time, and the second is by setting the message itself, then the expiration time of each message is different. If both methods are used at the same time, the value with the smaller expiration time shall prevail. When a message reaches its expiration date and has not been consumed, then that message becomes a dead letter message.

Queue settings: use the x-message-ttl parameter (in milliseconds) when the queue is declared

Single message setting: is the value of the expiration parameter that sets the message property (in milliseconds)

Delay queue: there is no delay queue in rabbitmq, but we can simulate the delay queue by setting the expiration time of the message and the dead letter queue. Consumers listen to queues bound by dead-letter switches rather than queues that send messages.

With the above basic knowledge, we meet the following requirements:

Requirements: the user creates an order in the system and automatically cancels the order if the user fails to pay for it after the expiration of the time.

Analysis:

1. In the above case, it is suitable to use a delay queue, so how to create a delay queue

2. Delay queues can be determined by expired messages + dead letter queues.

3. Expired messages are realized by setting the x-message-ttl parameter in the queue.

4. The Dead letter queue sets the x-dead-letter-exchange parameter to the queue when the queue is declared, and then declares that a queue is bound to the switch corresponding to x-dead-letter-exchange.

ConnectionFactory factory = new ConnectionFactory (); factory.setHost ("127.0.0.1"); factory.setPort (AMQP.PROTOCOL.PORT); factory.setUsername ("guest"); factory.setPassword ("guest"); Connection connection = factory.newConnection (); Channel channel = connection.createChannel (); / / declare a switch and queue that receives deleted messages String EXCHANGE_DEAD_NAME = "exchange.dead"; String QUEUE_DEAD_NAME = "queue_dead" Channel.exchangeDeclare (EXCHANGE_DEAD_NAME, BuiltinExchangeType.DIRECT); channel.queueDeclare (QUEUE_DEAD_NAME, false, null); channel.queueBind (QUEUE_DEAD_NAME, EXCHANGE_DEAD_NAME, "routingkey.dead"); String EXCHANGE_NAME = "exchange.fanout"; String QUEUE_NAME = "queue_name"; channel.exchangeDeclare (EXCHANGE_NAME, BuiltinExchangeType.FANOUT); Map arguments = new HashMap () / / uniformly set the expiration time of all messages in the queue arguments.put ("x-message-ttl", 30000); / / set the number of milliseconds in which no consumer accesses the queue, then delete the queue arguments.put ("x-expires", 20000); / / set the latest N messages of the queue. If more than N messages, the previous message will remove the arguments.put from the queue ("x-max-length", 4). / / set the maximum space for the contents of the queue. Beyond this threshold, delete the previous message arguments.put ("x-max-length-bytes", 1024); / / push the deleted message to the specified switch. Generally, x-dead-letter-exchange and x-dead-letter-routing-key need to set arguments.put ("x-dead-letter-exchange", "exchange.dead") at the same time. / / push the deleted message to the routing key arguments.put ("x-dead-letter-routing-key", "routingkey.dead") corresponding to the specified switch; / / set the priority of the message. The higher priority is consumed arguments.put ("x-max-priority", 10); channel.queueDeclare (QUEUE_NAME, false, arguments); channel.queueBind (QUEUE_NAME, EXCHANGE_NAME, "); String message =" Hello RabbitMQ: " For (int I = 1; I

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.