Shulou(Shulou.com)06/01 Report--

This article shows you how to achieve MQ without losing messages. The content is concise and easy to understand. It will definitely brighten your eyes. I hope you can gain something through the detailed introduction of this article.

Send tasks and messages through message queuing (MsgQueue,MQ). What if MQ is restarted? Can you guarantee that MQ will not lose information? Today, let's talk about MQ's message accessibility architecture and process. Without losing messages, what is the core direction of MQ architecture design? If the message is to be reached in MQ, there are two core design points in the architecture: (1) message landing, (2) message timeout, retransmission and confirmation.

In order to achieve the above two core points, what is the MQ architecture? The image above is a core architecture diagram of MQ, which can be divided into three parts: (1) sender-> left pink part; (2) MQ core cluster-> middle blue part; and (3) receiver-> right shit yellow part.

The pink sender consists of two parts: (1) the business caller and (2) the MQ-client-sender;, in which the latter provides the former with two core API:

(1) SendMsg (bytes [] msg)

(2) SendCallback ()

The blue MQ core cluster is divided into four parts:

(1) MQ-server

(2) zk

(3) db

(4) manage backend web

The yellow receiver also consists of two parts: (1) the service receiver and (2) the MQ-client-receiver;, in which the latter provides the former with two core API:

(1) RecvCallback (bytes [] msg)

(2) SendAck ()

MQ is a powerful tool for decoupling between systems. It can decouple the release subscribers. It decouples the upstream and downstream message delivery into two parts, such as the 1 arrow and 2 arrow in the architecture diagram:

Arrow 1: the sender delivers the message to MQ, the first half; the arrow 2:MQ delivers the message to the receiver, the second half

What is the core process of reliable delivery of MQ messages? Since MQ has split the message delivery into the first and second half, in order to ensure the reliable delivery of the message, the message must be delivered in both the first and second half. In the first half of MQ message delivery, the MQ-client-sender-to-MQ-server process is shown in figure 1-3 above: (1) MQ-client sends the message to MQ-server; voiceover: at this time, the business side calls API:SendMsg. (2) when the message is landed by MQ-server, it will be sent successfully; (3) MQ-server sends the reply to MQ-client; voiceover: call back the service API:SendCallback at this time.

In the second half of MQ message delivery, the process from MQ-server to MQ-client-receiver is shown in figure 4-6 above: (4) MQ-server sends the message to MQ-client; voiceover: call back the business API:RecvCallback at this time. (5) MQ-client reply reply to MQ-server; voiceover: at this time, the business side actively calls API:SendAck. (6) MQ-server receives ack, deletes messages that have been landed before, and completes reliable delivery of messages.

What if the message is lost? Message loss can occur in the first and second half of MQ message delivery. In order to ensure message reachability, MQ needs to time out and retransmit.

How to implement timeout and retransmission in the first half? If the 1, 2 or 3 of the first half of the MQ is lost or timed out, the timer in the MQ-client-sender will resend the message until it is expected to receive 3. If it is not received after N retransmissions, the SendCallback callback will fail. It should be noted that during this process, the MQ-server may receive multiple retransmissions of the same message.

How to implement timeout and retransmission in the second half? If the 4 or 5 or 6 of the second half of the MQ is lost or times out, the timer in the MQ-server will resend the message until 5 is received and 6 is successfully executed, a process that may resend the message many times. Voiceover: the exponential Backoff strategy is generally adopted, which is retransmitted every x seconds, 2x seconds, 4x seconds, and so on. It is important to note that MQ-client-receiver may also receive multiple retransmissions of the same message during this process. MQ is a sharp tool for decoupling between systems. In order to ensure that messages must be reached, MQ's architecture design direction is: (1) messages will be received first; (2) messages will be timed out, retransmitted, and confirmed to ensure that messages will arrive; the above is how MQ does not lose messages. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.