Shulou(Shulou.com)06/02 Report--

This article will explain in detail the process of investigating the causes of the failure of the big news about MQTT. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have some understanding of the relevant knowledge after reading this article.

Background

A chat server was set up in the group using MQTT protocol. When testing big messages (more than 5000 Chinese characters) the day before yesterday, the connection directly became unavailable, and all subsequent messages were not answered.

Server environment:

Netty: 4.1.32.Final

Using the MqttDecoder that comes with the Netty package

Client: Android

Investigation process

Since all the messages are printed in the log, I searched the server log first and found that there was no message content in the log.

Is it possible that the client did not send a very long message? Using tcpdump to catch the packet, it is found that the client sends it normally, and all the packet servers have already ack, but the subsequent server does not send back a response. It is guessed that the server failed to process the message in the case of a big message.

Tcpdump uses-nn to print out the ip and port, and-X to print the contents of the network packet, or you can use the-w option to save to a file, and then use tcpdump or wireshark to analyze

So I checked that the maximum payload,MQTT supported by MQTT is 256m, which is definitely not larger than that.

The package was grabbed at the server, and the confirmation message was received, but no confirmation message was returned.

Open the online debug and find that a message of type PUBLISH has been received, but the class of the message is not MqttPublishMessage and there is no data in payload, but there is an error message too large message: 56234 bytes in Message

Google, some netizens encountered the same problem, although MQTT is in C language in this problem.

Check the MqttDecoder and find that decoder has a maximum payload limit (the following is part of the code). The default constructor is called in the startup code, so the default maximum data length is 8092 bytes.

Public final class MqttDecoder extends ReplayingDecoder {private static final int DEFAULT_MAX_BYTES_IN_MESSAGE = 8092; public MqttDecoder () {this (DEFAULT_MAX_BYTES_IN_MESSAGE);} public MqttDecoder (int maxBytesInMessage) {super (DecoderState.READ_FIXED_HEADER); this.maxBytesInMessage = maxBytesInMessage } @ Override protected void decode (ChannelHandlerContext ctx, ByteBuf buffer, List out) throws Exception {switch (state ()) {case READ_FIXED_HEADER: try {mqttFixedHeader = decodeFixedHeader (buffer); bytesRemainingInVariablePart = mqttFixedHeader.remainingLength (); checkpoint (DecoderState.READ_VARIABLE_HEADER) / / fall through} catch (Exception cause) {out.add (invalidMessage (cause)); return;} case READ_VARIABLE_HEADER: try {final Result decodedVariableHeader = decodeVariableHeader (buffer, mqttFixedHeader); variableHeader = decodedVariableHeader.value If (bytesRemainingInVariablePart > maxBytesInMessage) {throw new DecoderException ("too large message:" + bytesRemainingInVariablePart + "bytes");} bytesRemainingInVariablePart-= decodedVariableHeader.numberOfBytesConsumed; checkpoint (DecoderState.READ_PAYLOAD); / / fall through} catch (Exception cause) {out.add (invalidMessage (cause)) Return;} case READ_PAYLOAD: try {final Result decodedPayload = decodePayload (buffer, mqttFixedHeader.messageType (), bytesRemainingInVariablePart, variableHeader) BytesRemainingInVariablePart-= decodedPayload.numberOfBytesConsumed; if (bytesRemainingInVariablePart! = 0) {throw new DecoderException ("non-zero remaining payload bytes:" + bytesRemainingInVariablePart + "(" + mqttFixedHeader.messageType () +')') } checkpoint (DecoderState.READ_FIXED_HEADER); MqttMessage message = MqttMessageFactory.newMessage (mqttFixedHeader, variableHeader, decodedPayload.value); mqttFixedHeader = null; variableHeader = null; out.add (message); break } catch (Exception cause) {out.add (invalidMessage (cause)); return;} case BAD_MESSAGE: / / Keep discarding until disconnection. Buffer.skipBytes (actualReadableBytes ()); break; default: / / Shouldn't reach here. Throw new Error ();} private MqttMessage invalidMessage (Throwable cause) {checkpoint (DecoderState.BAD_MESSAGE); return MqttMessageFactory.newInvalidMessage (mqttFixedHeader, variableHeader, cause);}}

The reason for the long message has been found, and there is only one question left: why can't follow-up messages, including ping messages, be sent any more? After looking at the code, this has something to do with the parent class ReplayingDecoder of MqttDecoder. Check that the source code has a detailed description of the class. If the payload exceeds the maximum limit when reading variable length headers, then throw an exception directly. The extraction code is as follows:

Case READ_VARIABLE_HEADER: try {final Result decodedVariableHeader = decodeVariableHeader (buffer, mqttFixedHeader); variableHeader = decodedVariableHeader.value; if (bytesRemainingInVariablePart > maxBytesInMessage) {throw new DecoderException ("too large message:" + bytesRemainingInVariablePart + "bytes");} bytesRemainingInVariablePart-= decodedVariableHeader.numberOfBytesConsumed; checkpoint (DecoderState.READ_PAYLOAD); / / fall through} catch (Exception cause) {out.add (invalidMessage (cause); return;})

In exception handling, the invalidMessage method is called, which sets the state to DecoderState.BAD_MESSAGE, where all bytes are discarded directly.

Case BAD_MESSAGE: / / Keep discarding until disconnection. Buffer.skipBytes (actualReadableBytes ()); break

In other words, none of the subsequent messages will enter the business processing logic, and this long connection is scrapped.

Solution

The client limits and splits long messages to ensure that a single message does not exceed the maximum limit.

The server increases the maximum load length, and MqttDecoder provides a constructor (not recommended, which increases the server's processing time and memory burden)

This is the end of the process of investigating the causes of the failure of the big news about MQTT. I hope the above content can be of some help to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.