Shulou(Shulou.com)06/01 Report--

TCP glue unpack and Netty code example analysis, I believe that many inexperienced people do not know what to do, so this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

TCP is a "stream" protocol. A stream is a string of data that has no boundaries. You can think of the running water in the river, which is connected, and there is no dividing line between them. The bottom layer of TCP does not understand the specific meaning of the upper layer business data, and it will divide packets according to the actual situation of the TCP buffer, so in business, it is considered that a complete packet may be split into multiple packets by TCP to send, or multiple small packets may be encapsulated into a large packet to send, which is the so-called TCP sticking and unpacking problem.

Reasons for TCP sticking or unpacking

The data written by the application is larger than the size of the socket buffer, which will cause unpacking.

The data written by the application is less than the size of the socket buffer, and the network card sends the data written by the application multiple times to the network, which will cause sticking packets.

Perform TCP segmentation of MSS (maximum message length) size, and unpacking will occur when the TCP message length-TCP header length > MSS.

The receiving method does not read the socket buffer data in time, which will cause sticking.

The form of unpacking and sticking.

The first case: the receiver normally receives two data packets, that is, there is no phenomenon of unpacking and sticking packets, which is beyond the scope of this article.

The second case: the receiver only receives one packet, because TCP will not lose the packet, so this packet contains the information of the two packets sent by the sender, this phenomenon is called sticking packet. In this case, because the receiver does not know the boundary between the two packets, it is difficult for the receiver to deal with.

The third situation: there are two forms of this situation, such as the following figure. The receiver receives two packets, but the two packets are either incomplete or have an extra piece, that is, unpacking and sticking occurs. If these two cases are not treated specially, it is also difficult for the receiver to deal with.

The solution of sticking and unpacking

The sender adds a packet header to each packet, and the header should at least contain the length of the packet, so that after receiving the data, the receiver knows the actual length of each packet by reading the length field of the header.

The sender encapsulates each packet as a fixed length (if it is not enough, it can be filled with zero padding), so that the receiver automatically splits each packet each time it reads fixed-length data from the receive buffer.

You can set a boundary between packets and add special symbols (such as carriage returns) so that the receiver can split different packets through this boundary.

Code examples in Netty

Netty encapsulates the NIO of JDK and is an asynchronous event-driven network application framework for the rapid development of maintainable high-performance servers and clients. JDK native NIO is not used in general development for the following reasons:

Using NIO that comes with JDK needs to know too many concepts, programming is complicated, and accidentally bug flies across.

The underlying IO model of Netty can be switched at will, and all these only need to make minor changes, change the parameters, and Netty can be changed directly from NIO model to IO model.

Netty's own mechanisms such as unpacking and exception detection separate you from the onerous details of NIO, so that you only need to care about business logic.

Netty solves many bug of JDK, including empty polling.

The bottom layer of Netty makes a lot of small optimizations for threads and selector, and the well-designed reactor thread model achieves very efficient concurrency processing.

It comes with a variety of protocol stacks that allow you to deal with any general protocol without having to do it yourself.

The Netty community is active, mailing list or issue whenever you encounter problems.

Netty has experienced extensive online verification of major rpc frameworks, message middleware and distributed communication middleware, and its robustness is extremely powerful.

Therefore, this article chooses to demonstrate the codec code of Netty.

In Netty, we define the inheritance class of MessageToByteEncoder, rewrite its encode function, and customize the encoder.

Public class SocketEncoder extends MessageToByteEncoder {@ Override protected void encode (ChannelHandlerContext channelHandlerContext, NetPacket msg, ByteBuf byteBuf) throws Exception {byte body [] = msg.getBody (); int packetLen = body.length; / / set the packet length first, and then write binary data byteBuf.writeInt (packetLen); byteBuf.writeBytes (body);}}

In Netty, we define the inheritance class of ByteToMessageDecoder, override its decode function, and customize the decoder.

Public class SocketDecoder extends ByteToMessageDecoder {@ Override void decode (ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List list) throws Exception {int bufLen = byteBuf.readableBytes (); / / solve the sticky packet problem (less than the length of a packet header) / / 4 bytes is that an int is used in the message to represent the message length if (bufLen < 4) {return } / / Mark the location of the current readIndex byteBuf.markReaderIndex (); int packetLength = byteBuf.readInt ()

/ / if the length of the message body read is less than the length of the message we sent, then resetReaderIndex. Reset the read index and continue to receive if (byteBuf.readableBytes () < packetLength) {/ / for use with markReaderIndex. Reset readIndex to the place of mark byteBuf.resetReaderIndex (); return;}

NetPacket netPacket = new NetPacket (); netPacket.setPacketLen (packetLength); / / the length of the transmitted data meets our requirements: byte body [] = new byte [packetLength]; byteBuf.readBytes (body); netPacket.setBody (body); list.add (netPacket);}} after reading the above, have you mastered the method of example analysis of TCP gluing and unpacking package and Netty code? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.