Shulou(Shulou.com)06/01 Report--

This article is about how to analyze the track of RocketMQ messages. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it with the editor.

RocketMQ message trajectory, which mainly tracks the trajectory of message delivery and message consumption, that is, records the log of each processing step of the message in detail. In terms of design, at least the following three core problems need to be solved:

Message track data format

The message track of RocketMQ4.5 version mainly records the following information:

TraceType

Trace type. Available values: Pub (message sending), SubBefore (message pulling to the client, before executing the business-defined consumption logic), SubAfter (after consumption).

TimeStamp

Current timestamp.

RegionId

The region where the broker is located is ID, which is taken from BrokerConfig#regionId.

GroupName

Group name. The name of the producer group when traceType is Pub, and the consumer group name if traceType is subBefore or subAfter.

RequestId

When traceType is subBefore or subAfter, the request Id on the consumer side.

Topic

Message subject.

MsgId

The message is unique ID.

Tags

Message tag.

Keys

The message index key, according to which the message can be quickly retrieved.

StoreHost

When the trace type is PUB, the IP; trace type of the Broker server that stores the message is subBefore, and when subAfter is consumer IP.

BodyLength

The length of the message body.

CostTime

It takes time.

MsgType

Type of message. Available values: Normal_Msg (normal message), Trans_Msg_Half (pre-submitted message), Trans_msg_Commit (submitted message), Delay_Msg (delayed message).

OffsetMsgId

The message offset ID, which contains the ip of the broker and the offset.

Success

It was sent successfully.

ContextCode

Consumption status code. Available value: SUCCESS,TIME_OUT,EXCEPTION,RETURNNULL,FAILED.

Record the message track

The two core themes of message middleware: message sending and message consumption, the core carrier is the message. The message trajectory (message flow) mainly records when the message is sent to which Broker, how long it takes to send the message, and what is consumed by which consumer. The track of recording messages is mainly concentrated before and after the message is sent and before and after the message consumption, which can be done through the Hook mechanism of RokcetMQ. Hook functions are defined through the following two interfaces.

By implementing the above two interfaces, the message trajectory can be recorded before and after message sending and message consumption. in order not to significantly increase the delay of message sending and message consumption, it is best to use asynchronous sending mode to record message trajectory. How to store message track data

The question of what messages need to be stored and when to record the message tracks is solved, so the next step is to think about where to store the message tracks. Storing in the database or other media will aggravate the message middleware and make it rely on external components. The best choice is to store the message track data in the Broker server and store the message trajectory data to the Broker server as a message.

Since message tracks are stored on the Broker server as messages, how can the Topic that stores the message tracks be determined? RocketMQ provides two ways to define the Topic of the message trace.

System default Topic

If the traceTopicEnable configuration of Broker is set to true, it means that the name of the topic created on the Broker is: RMQ_SYS_TRACE_TOPIC, the number of queues is 1, and the default value is false, which means that the Broker does not host the topic customized by the system to store message tracks.

Custom Topic

When creating a message producer or message consumer, you can customize the Topic name used to record the message track through parameters. However, it is important to note that only one message track Topic is supported in the rokcetmq console (rocketmq-console), so customizing Topic may not be a best practice at this stage. It is recommended to use the default Topic of the system.

Usually, in order to avoid mixing message trace data with normal business data, officials suggest that a new machine should be added to the Broker cluster, and message trace tracking should only be enabled on this machine. In this way, the message trace data in the cluster will only be sent to this Broker server, which will not increase the load of the original business Broker in the cluster.

The above is how to analyze the trajectory of RocketMQ messages. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.