Shulou(Shulou.com)06/01 Report--

TCP sticky unpack problem and what is the solution in Netty, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can gain something.

Those who are familiar with TCP programming know that both the server and the client need to consider the underlying sticking / unpacking mechanism of TCP when we read or send messages.

TCP sticking / unpacking problems often do not occur during functional testing, but once the concurrent pressure comes up, or after sending a big message, it is easy to have the sticking / unpacking problem. If the code is not considered, there will often be decoding errors or errors, resulting in the program can not work properly.

In this article, we first briefly understand the basics of TCP gluing / unpacking, and then take a look at how Netty solves this problem.

TCP sticky package / unpacking TCP sticky package / unpacking problem description

TCP is a streaming protocol. The so-called stream is a string of data without boundaries. The bottom layer of TCP does not understand the specific meaning of the upper layer (such as HTTP protocol) business data, it will divide packets according to the actual situation of TCP buffer, so in business, it is considered that a complete packet may be split into multiple packets by TCP to send, or multiple small packets may be encapsulated into a large packet to send, which is the so-called TCP sticking and unpacking problem. We can use the following example diagram to illustrate the problems of TCP sticking and unpacking.

Suppose the client sends two packets DI and D2 to the server in turn. Because the number of bytes read by the server at one time is uncertain, there may be the following four situations.

1. The server reads two independent packets, D1 and D2, without sticking and unpacking. The server receives two packets at a time, DI and D2 are glued together, which is called TCP sticky packet; 3. The server reads two packets twice, the first time it reads the complete DI packet and part of the D2 packet, the second time it reads the rest of the D2 packet, which is called TCP packet unpacking; 4. The server reads two packets twice, the first time it reads part of the D1 packet, and the second time it reads the rest of the D1 packet and the whole D2 packet.

If the TCP receiving window of the server is very small at this time, and the packets DI and D2 are relatively large, the fifth possibility is likely to occur, that is, the server divides multiple times to receive the D1 and D2 packets completely, and the packets are unpacked many times during the period.

The cause of TCP sticking / unpacking

There are three causes of the problem, which are as follows:

1. The byte size written by the application write exceeds the socket send buffer size; 2. TCP segmentation with MSS size; 3. The payload of the Ethernet frame is greater than MTU for IP slicing.

Strategies for solving the problem of gluing and unpacking

Because the bottom TCP cannot understand the upper layer business data, there is no guarantee that the packets will not be split and reassembled at the bottom. This problem can only be solved through the upper layer application protocol stack design. According to the solutions of the mainstream protocols in the industry, it can be summarized as follows.

1. Fixed message length, for example, the size of each message is a fixed length of 200 bytes, if not, fill in the blanks; 2. Special characters such as "carriage return newline" are used at the end of the packet as a sign of the end of the message, such as FTP protocol, which is widely used in text protocols. 3. The message is divided into header and message body, and a length field Len is defined in the header to identify the total length of the message. 4. More complex application layer protocols.

After introducing the basics of TCP sticking / unpacking, let's take a look at how Netty uses a series of "half-packet decoders" to solve the TCP sticking / unpacking problem.

Netty Decoder solves the problem of TCP sticking and unpacking

According to the above sticking and unpacking problem solving strategy, Netty provides the corresponding decoder implementation. With these decoders, users do not need to manually decode the read messages, nor do they need to consider the sticking and unpacking of TCP packets.

Principle Analysis of LineBasedFrameDecoder and StringDecoder

In order to solve the half-package read-write problem caused by TCP sticking / unpacking, Netty provides a variety of codecs by default to deal with half-package. as long as you are proficient in the use of these libraries, TCP gluing and unpacking problems will become very easy, and you don't even need to care about them, which is unmatched by other NIO frameworks and JDK native NIO API. For users, as long as the Handler that supports half-packet decoding is added to the ChannelPipeline object, there is no need to write additional code, and it is very easy to use.

/ / sample code, where socketChannel is a SocketChannel object socketChannel.pipeline () .addLast (new LineBasedFrameDecoder (1024)); socketChannel.pipeline () .addLast (new StringDecoder ())

How LineBasedFrameDecoder works is that it iterates through the readable bytes in ByteBuf in turn to see if there are\ n or\ r\ n, and if so, this position is the end position, and the bytes in the range from the readable index to the end position form a line. It is a decoder marked with a newline character, which supports decoding with or without a Terminator, and supports the configuration of the maximum length of a single line. If no newline character is found after continuously reading the maximum length, an exception is thrown and the previously read exception stream is ignored.

The function of StringDecoder is very simple, which is to convert the received object into a string, and then continue to call the following Handler. The combination of LineBasedFrameDecoder and StringDecoder is a line-switched text decoder, which is designed to support gluing and unpacking of TCP.

Other decoders

In addition to LineBasedFrameDecoder, there are two commonly used decoders DelimiterBasedFrameDecoder and FixedLengthFrameDecoder, the former can automatically decode "messages marked with delimiters", and the latter can automatically decode fixed-length messages. The method is also the same as the previous sample code, combined with the string decoder StringDecoder, it is easy to complete the automatic decoding of many messages.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.