Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to solve the problem of sticking and unpacking in Netty. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

TCP gluing and unpacking

TCP is a "stream" protocol. A stream is a string of data that has no boundaries. The bottom layer of TCP does not understand the specific meaning of the upper layer business data, and it will divide packets according to the actual situation of the TCP buffer, so in business, it is considered that a complete packet may be split into multiple packets by TCP to send, or multiple small packets may be encapsulated into a large packet to send, which is the so-called TCP sticking and unpacking problem.

As shown in the figure, assuming that the client sends two packets D1 and D2 to the server, because the number of bytes read by the server at one time is uncertain, there may be the following four situations.

The server reads two separate packets, D1 and D2, without sticking and unpacking.

The server receives two packets at a time. D1 and D2 are glued together, which is called TCP sticky packet.

The server reads two packets twice, the first time it reads part of the complete D1 packet and D2 packet, and the second time it reads the rest of the D2 packet, which is called TCP unpacking.

The server reads two packets twice, the first time it reads the part of the D1 packet, D1room1, and the second time it reads the whole packet of the remaining contents of the D1 packet, D1room2 and D2.

If the TCP receiving window of the server is very small and the packets D1 and D2 are relatively large, the fifth possibility is likely to occur, that is, the server divides multiple times to receive the D1 and D2 packets completely, and the packets are unpacked many times during this period.

The causes of sticking and unpacking of TCP

Data from sender to receiver needs to pass through the buffer of the operating system, and the main reason for sticking and unpacking is on this buffer. Sticky packet can be understood as buffer data accumulation, resulting in multiple request data glued together, while unpacking can be understood as sending data larger than the buffer, split processing.

In detail, there are three main reasons for sticking and unpacking:

The byte size written by the application write is larger than the socket send buffer size.

Perform TCP segmentation of MSS size

The payload of the Ethernet frame is greater than MTU for IP slicing.

The solution of gluing and unpacking

Because the bottom TCP cannot understand the upper layer business data, there is no guarantee that the packets will not be split and reassembled at the bottom. This problem can only be solved through the upper layer application protocol stack design. According to the solutions of the mainstream protocols in the industry, it can be summarized as follows.

The length of the message is fixed, and after the message of length and fixed length LEN is read, it is considered that a complete information has been read.

Use carriage return newline character as message Terminator

The special delimiter is used as the end mark of the message, and the carriage return newline character is a special end delimiter.

Identify the total length of the message by defining a length field in the header

The solution of gluing and unpacking in Netty

For the solution of sticking and unpacking described in the previous section, the problem of unpacking is relatively simple. Users can define their own encoders to deal with, and Netty does not provide the corresponding components. For the problem of sticking packets, due to the complexity of unpacking and the tedious processing of code comparison, Netty provides four decoders to solve the problem, which are as follows:

Fixed-length unpacker FixedLengthFrameDecoder. Each application layer packet is split to a fixed-length size.

Line unpacker LineBasedFrameDecoder, each application layer packet is split with a newline character as a delimiter

Delimiter DelimiterBasedFrameDecoder. Each application layer packet is split by a custom delimiter.

The LengthFieldBasedFrameDecoder, which is based on the packet length, takes the length of the application layer packet as the basis for splitting the application layer packet at the receiver.

Unpack the packet according to the size of the application layer packet.

There is a requirement for this unpacking device, that is, the length of the packet contained in the application layer protocol.

The above decoders only need to be added to the responsibility chain of Netty. In most cases, these four decoders can be satisfied. Of course, in addition to the above four decoders, users can also customize their own decoders for processing. For more information, please see the following code example:

/ / Server main program

Public class XNettyServer {

Public static void main (String [] args) throws Exception {

/ / accept thread pool for processing connections

NioEventLoopGroup acceptGroup = new NioEventLoopGroup ()

/ / read io thread pool for processing data

NioEventLoopGroup readGroup = new NioEventLoopGroup ()

Try {

ServerBootstrap serverBootstrap = new ServerBootstrap ()

ServerBootstrap

.group (acceptGroup, readGroup)

.channel (NioServerSocketChannel.class)

.childHandler (

New ChannelInitializer () {

@ Override

Protected void initChannel (SocketChannel ch) throws Exception {

ChannelPipeline pipeline = ch.pipeline ()

/ / add a decoder

Pipeline.addLast (new XDecoder ())

/ / print out the content handdler

Pipeline.addLast (new XHandler ())

}

});

System.out.println ("started successfully, port 7777")

ServerBootstrap.bind (7777). Sync (). Channel (). CloseFuture (). Sync ()

} finally {

AcceptGroup.shutdownGracefully ()

ReadGroup.shutdownGracefully ()

}

}

}

/ / Decoder

Public class XDecoder extends ByteToMessageDecoder {

Static final int PACKET_SIZE = 220

/ / use the request message that has not been processed in the coming time.

ByteBuf tempMsg = Unpooled.buffer ()

/ * *

* @ param ctx

* @ param in requested data

* @ param out retains the results of the split of the glued messages

* @ throws Exception

, /

@ Override

Protected void decode (ChannelHandlerContext ctx, ByteBuf in, List out) throws Exception {

System.out.println (Thread.currentThread () + "received a packet with a length of" + in.readableBytes ()).

/ / merge message

ByteBuf message = null

Int tmpMsgSize = tempMsg.readableBytes ()

/ / merge if the last remaining request message is temporarily stored

If (tmpMsgSize > 0) {

Message = Unpooled.buffer ()

Message.writeBytes (tempMsg)

Message.writeBytes (in)

System.out.println ("merge: the remaining length of the previous packet is:" + tmpMsgSize + ", and the merged length is:" + message.readableBytes ())

} else {

Message = in

}

Int size = message.readableBytes ()

Int counter = size / PACKET_SIZE

For (int I = 0; I < counter; iTunes +) {

Byte [] request = new byte [pack _ SIZE]

/ / 220 bytes of data are read from the total message at a time

Message.readBytes (request)

/ / put the split result into the out list and leave it to the later business logic to process.

Out.add (Unpooled.copiedBuffer (request))

}

/ / save the extra messages

/ / first message: I + temporary storage

/ / second message: 1 and the first time

Size = message.readableBytes ()

If (size! = 0) {

System.out.println ("extra data length:" + size)

/ / put the remaining data into tempMsg for temporary storage

TempMsg.clear ()

TempMsg.writeBytes (message.readBytes (size))

}

}

}

/ / processor

Public class XHandler extends ChannelInboundHandlerAdapter {

@ Override

Public void channelReadComplete (ChannelHandlerContext ctx) throws Exception {

Ctx.flush ()

}

@ Override

Public void channelRead (ChannelHandlerContext ctx, Object msg) throws Exception {

ByteBuf byteBuf = (ByteBuf) msg

Byte [] content = new byte [byteBuf.readableBytes ()]

ByteBuf.readBytes (content)

System.out.println (Thread.currentThread () + ": final print" + new String (content))

(ByteBuf) msg) .release ()

}

@ Override

Public void exceptionCaught (ChannelHandlerContext ctx, Throwable cause) throws Exception {

Cause.printStackTrace ()

Ctx.close ()

}

} about how to solve the problem of sticking and unpacking in Netty is shared here. I hope the above content can be helpful to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.