Shulou(Shulou.com)06/02 Report--

This article mainly explains "what are the advanced features of RabbitMQ". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what are the advanced features of RabbitMQ".

1. How to ensure 100% successful delivery of messages 1: message storage and marking message status 1) the sender stores the business data to BIZ_DB and sets the message to MSG_DB,status to 0. As long as one operation fails, no further operation is carried out. If this step fails, the next operation is not carried out. Mechanism for rapid failure 2) the sender sends a message to the MQ 3) the MQ confirms to the sender after receiving the message 4) the sender monitors the acknowledgement message and sets the message status to 1, which is the logic of normal success. 5) the sender has a distributed scheduled task to scan the message with a timeout of 0 status in the table, for example, the timeout period can be set to 2min6) for a message with a status of 0 Resend 7) set the maximum number of retries, such as Retry_count > 3, set the message status to 2 Note: 1) Business data and messages can fall into one database. For two libraries and for small-scale design applications, transactions can be opened to ensure that the data sources are consistent. In the case of large-scale and high concurrency, I have not heard that any large Internet companies have opened transactions and adopted compensation mechanism. 2) disadvantages: step 1 needs to persist the database twice. 3) distributed scheduled tasks to ensure that there is only one task at a time to grab the first reliable delivery of DB. Is it suitable for high concurrency scenarios? The first method needs to be stored twice, and there is great pressure on the database in the case of high concurrency. 2. How to ensure 100% successful delivery of messages 2: delayed delivery of messages, second confirmation, callback check (receiving huge amounts of data). The purpose of scheme 2 is to reduce the number of database persistence operations, such as order scenarios. The key point is not 100% success, the key point is performance, which can withstand high concurrency. For core links, reduce database operations, UpStream upstream service producer, DownStream downstream service consumer, Callback callback service 1) Business data is stored in the database only once, two messages are generated at one time, and then the first MQ message is sent to the message middleware. Note: the message must be sent after the business data is stored in the database. Internet companies do not add transactions, transactions seriously affect performance 2) the second MQ message is delayed for a period of time, delayed delivery, such as re-sending after 2min 3) consumers monitor and consume messages 4) after consumption is completed, consumers send confirmation messages to MQ The confirmation message is generated by the consumer. 5) the Callback service listens for the confirmation message. Callback knows that the downstream has been successfully processed. Callback stores the message. 6) Callback listens for the delayed delivery message, checks the MSG DB database, and finds that the message has been successfully processed. 7) if there is no such message in MSG DB before, Callback will RPC call UpStream to resend the MQ message Note: 1) it must be after the business data is stored in the database. Send messages 2) without transactions, transactions will cause serious performance bottlenecks 3) Callback is only a compensation service, it is not on the core link

3. The concept of idempotency and the idempotency of mainstream solutions in the industry: the characteristic of an idempotent operation is that the impact of arbitrary execution is the same as that of a single execution. We can learn from the optimistic locking mechanism of the database, such as executing a SQL statement to update inventory: use the version number to control it, such as elaticsearchupdate t_reps set count = count-1. Version = version + 1 where version = 1 idempotent guarantee on the consumer side how to avoid the problem of repeated consumption of messages during the business peak period when massive orders are generated? The realization of idempotency on the consumer side means that our messages will never be consumed many times, even if we receive many of the same mainstream idempotent operations in the industry: 1) unique ID + fingerprint code mechanism, using database primary key to remove duplicate unique ID + fingerprint code mechanism, fingerprint codes can be fields generated according to the business, such as timestamps or fields returned by the bank. Using database primary key to remove duplicates select count (1) from t_order where id = unique ID + fingerprint code, if 1 indicates that the benefits have been consumed before: simple disadvantages: performance bottleneck solution with database writing under high concurrency: follow up with ID to split database and table for algorithm routing 2) use the atomic characteristics of Redis to realize idempotency using Redis, problems to be considered Using redis's setnx command 1, do we need to store the data in the database? if so, the key problem to be solved is how to achieve the consistency between the database and the cache? 2. If the database is not removed, then it is stored in the cache. How to set the strategy of timing synchronization? 4. Confirm acknowledgement messages understand the Confirm message acknowledgement mechanism: message acknowledgement means that after the producer delivers the message, if the Broker receives the message It will give us a response from the producer. The producer receives the reply to determine whether the message is normally sent to Broker, which is also the core guarantee for the reliable delivery of the message! How to realize the Confirm confirmation message: 1) turn on the confirmation mode on channel, channel.confirmSelect () 2) add monitoring, addConfirmListener on channel, listen for the return result of success or failure, resend the message according to the specific result, or log and other subsequent processing.

5. Return message mechanism Return Listener is used to deal with some non-routable messages. Our message producer, by specifying an Exchange and Routing Key, sends the message to a queue, and then our consumer listens to the queue for consumption processing! But in some cases, if the current Exchange does not exist or the specified route key is not available when we send the message, and we need to listen for this kind of unreachable message, we need to use Return Listener to have an attribute Mandatory in the underlying API. If it is true, the listener will receive the message that the route is unreachable, and then process it later. If it is false, Then the broker side automatically deletes the message. 6. Consumer custom listener public class MyConsumer extends DefaultConsumer {public MyConsumer (Channel channel) {super (channel) } @ Override public void handleDelivery (String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte [] body) throws IOException {System.err.println ("- consume message-"); System.err.println ("consumerTag:" + consumerTag) System.err.println ("envelope:" + envelope); System.err.println ("properties:" + properties); System.err.println ("body:" + new String (body));} 7. Consumer-side current restrictions (focus) what is consumer-side current restrictions? Suppose the RabbitMQ server has tens of thousands of unprocessed messages, and a huge amount of messages are all pushed in an instant, but our single client cannot process so much data at the same time! there are two post-signing modes for RabbitMQ consumers: automatic signing and manual signing (recommended) RabbitMQ provides a QoS service quality assurance function, that is, under the premise of non-automatic confirmation of messages (setting non-automatic signing) If a certain number of messages (by setting the value of Qos based on consume or channel) are not acknowledged, new messages are not consumed. The implementation of this method is channel.basicQos (0,1, false) on the consumer side. Void BasicQos (uint prefetchSize,ushort prefetchCount,bool global) prefetchSize: set the size of listening messages to 0 without limiting prefetchCount: it will tell RabbitMQ not to push more than N messages to a consumer at the same time, that is, once there are N messages without ack, the consumer will drop the block until there is a message ackglobal:true\ false. To put it simply, whether to apply the above settings to channel. That is, whether the above limit is channel level or consumer level / / 1 the first thing about current restriction is that autoAck is set to falsechannel.basicQos (0,1, false) Channel.basicConsume (queueName, false, new MyConsumer (channel)); public class MyConsumer extends DefaultConsumer {private Channel channel; public MyConsumer (Channel channel) {super (channel); this.channel = channel } @ Override public void handleDelivery (String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte [] body) throws IOException {System.err.println ("- consume message-"); System.err.println ("consumerTag:" + consumerTag) System.err.println ("envelope:" + envelope); System.err.println ("properties:" + properties); System.err.println ("body:" + new String (body)); / / Core code. Ack,false does not support batch signing channel.basicAck (envelope.getDeliveryTag (), false). }} 8. The consumer ACK and the manual ACK and NACK consumers who return to the queue are returned to the queue in order to re-deliver the message to Broker for the messages that have not been successfully processed. Generally, in practical applications, we will turn off and return to the queue, that is, set to false.9. TTL queue / message TTL is the abbreviation of Time To Live, that is, the expiration time of RabbitMQ support messages. You can specify the expiration time of RabbitMQ support queues when messages are sent, starting from the time the messages are queued, as long as the timeout configuration of the queue is exceeded. Then the message is automatically cleared 10. Dead letter queue Private message queue: DLX Dead Letter Exchange uses DLX. When a message becomes a dead letter (dead message) in one queue, it can be re-publish to another Exchange, this Exchange is a dead letter queue. Several situations in which a message becomes a dead letter 1) the message is rejected 2) the message TTL expires 3) the queue reaches the maximum length DLX is also a normal Exchange, no different from the general Exchange, it can be specified on any queue, which is actually setting the properties of a queue. When there is a dead letter in this queue, RabbitMQ will automatically republish the message to the set Exchange, and then be routed to another queue, which can listen for messages in this queue to be processed accordingly. Thank you for reading, the above is the content of "what are the advanced features of RabbitMQ". After the study of this article, I believe you have a deeper understanding of what the advanced features of RabbitMQ are, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.