Shulou(Shulou.com)06/03 Report--

The article has been authorized by the author

Click "read the original text" in the lower left corner at the end of the text to jump to the original address.

From the beginning of the TCP connection to the end of the connection, data may be transferred multiple times, that is, multiple messages may be transmitted between the server and the client during the connection. Ideally, every time one party sends a message, the other party immediately receives one, that is, a write corresponds to a read. However, reality does not always follow the script.

Excerpts from MINA official documents:

TCP guarantess delivery of all packets in the correct order. But there is no guarantee that one write operation on the sender-side will result in one read event on the receiving side. One call of IoSession.write (Object message) by the sender can result in multiple messageReceived (IoSession session, Object message) events on the receiver; and multiple calls of IoSession.write (Object message) can lead to a single messageReceived event.

Excerpts from Netty official documents:

In a stream-based transport such as TCP/IP, received data is stored into a socket receive buffer. Unfortunately, the buffer of a stream-based transport is not a queue of packets but a queue of bytes. It means, even if you sent two messages as two independent packets, an operating system will not treat them as two messages but as just a bunch of bytes. Therefore, there is no guarantee that what you read is exactly what your remote peer wrote.

The meaning of the above two paragraphs is the same: TCP is a byte-stream-based protocol, which can only guarantee that the byte order of the data sent by one party is the same as that received by the other party, but it does not guarantee that every time one party sends a message, the other party will receive a complete message. It is possible to send two messages to merge it into one, or to send one to split it into two. So Demo in the previous article (Netty, MINA, and Twisted learn Series 01: implementing a simple TCP server) is a false demonstration. However, this problem is still difficult to test when the server and client are on the same machine or when the network speed is very good, such as the local area network.

To take a simple example (this example comes from the official Netty documentation): the message sender sent three strings:

But the receiver may receive something like this:

Then the problem is so serious that the receiver cannot separate the three messages and cannot interpret them. In response, the official documentation of MINA provides the following solutions:

1 、 use fixed length messages

Use fixed-length messages. For example, if each length is 4 bytes, then when receiving it, it can be split by 4 bytes each.

2 、 use a fixed length header that indicates the length of the body

The length (number of bytes) of the Body is specified in a fixed-length Header,Header to place the contents of the information in the Body. For example, if the length of the Body specified in the Header is 100bytes, then the 100byte after the Header is the Body, that is, the content of the message, and the 100byte Body is followed by the Header of the next message.

3. Using a delimiter; for example many text-based protocols append a newline (or CR LF pair) after every message

Use delimiters. For example, many agreements with text content add a newline character (CR LF, that is, "\ r\ n") to each message, that is, one message at a time. Of course, you can also use other special symbols as delimiters, such as commas, semicolons, and so on.

Of course, in addition to the three options mentioned above, there are other options. Some agreements may also use many of the above options at the same time. For example, in the HTTP protocol, the Header part uses CR LF line feeds to distinguish each Header, while Header uses Content-Length to specify the number of Body bytes.

Next, the message is segmented by the newline character CR LF with the related API implementations of MINA, Netty, and Twisted, respectively.

MINA

MINA can use ProtocolCodecFilter to send and receive binary data processing, how to process depends on ProtocolCodecFactory or ProtocolEncoder, ProtocolDecoder, after processing in IoHandler messageReceived event function to get the message is no longer IoBuffer, but you want other types, can be strings, Java objects. Here you can use TextLineCodecFactory, an implementation class of ProtocolCodecFactory, to implement CR LF splitting messages.

Public class TcpServer {

Public static void main (String [] args) throws IOException {

IoAcceptor acceptor = new NioSocketAcceptor ()

/ / add a Filter to split the content received and sent according to "\ r\ n"

Acceptor.getFilterChain () .addLast (codec)

New ProtocolCodecFilter (new TextLineCodecFactory (Charset.forName ("UTF-8"), "\ r\ n", "\ r\ n")

Acceptor.setHandler (new TcpServerHandle ())

Acceptor.bind (new InetSocketAddress (8080))

}

}

Class TcpServerHandle extends IoHandlerAdapter {

@ Override

Public void exceptionCaught (IoSession session, Throwable cause)

Throws Exception {

Cause.printStackTrace ()

}

/ / New data received

@ Override

Public void messageReceived (IoSession session, Object message)

Throws Exception {

/ / receive the data from the client. What is received here is no longer an IoBuffer type, but a string.

String line = (String) message

System.out.println ("messageReceived:" + line)

}

@ Override

Public void sessionCreated (IoSession session) throws Exception {

System.out.println ("sessionCreated")

}

@ Override

Public void sessionClosed (IoSession session) throws Exception {

System.out.println ("sessionClosed")

}

}

Netty

Netty is similar in design to MINA in that it needs to add some ChannelHandler to ChannelPipeline to process the raw data. Here, the received data is split by lines with LineBasedFrameDecoder, and the StringDecoder converts the data from bytecode to string. Similarly, after the received data is processed, in the channelRead event function, the msg parameter is no longer ByteBuf but String.

Public class TcpServer {

Public static void main (String [] args) throws InterruptedException {

EventLoopGroup bossGroup = new NioEventLoopGroup ()

EventLoopGroup workerGroup = new NioEventLoopGroup ()

Try {

ServerBootstrap b = new ServerBootstrap ()

B.group (bossGroup, workerGroup)

.channel (NioServerSocketChannel.class)

.childHandler (new ChannelInitializer () {

@ Override

Public void initChannel (SocketChannel ch)

Throws Exception {

ChannelPipeline pipeline = ch.pipeline ()

/ / LineBasedFrameDecoder splits messages by line

Pipeline.addLast (new LineBasedFrameDecoder (80))

/ / then convert it to a string by UTF-8 encoding

Pipeline.addLast (new StringDecoder (CharsetUtil.UTF_8))

Pipeline.addLast (new TcpServerHandler ())

}

});

ChannelFuture f = b.bind (8080) .sync ()

F.channel (). CloseFuture (). Sync ()

} finally {

WorkerGroup.shutdownGracefully ()

BossGroup.shutdownGracefully ()

}

}

}

Class TcpServerHandler extends ChannelInboundHandlerAdapter {

/ / New data received

@ Override

Public void channelRead (ChannelHandlerContext ctx, Object msg) {

/ / after StringDecoder, the type of msg is no longer ByteBuf but String.

String line = (String) msg

System.out.println ("channelRead:" + line)

}

@ Override

Public void channelActive (ChannelHandlerContext ctx) {

System.out.println ("channelActive")

}

@ Override

Public void channelInactive (ChannelHandlerContext ctx) {

System.out.println ("channelInactive")

}

@ Override

Public void exceptionCaught (ChannelHandlerContext ctx, Throwable cause) {

Cause.printStackTrace ()

Ctx.close ()

}

}

Twisted

The design of Twisted is not quite the same as that of the above two, so it is not the same to implement message segmentation. The class TcpServerHandle that handles events no longer inherits Protocol, but inherits LineOnlyReceiver, a subclass of Protocol. The event method that receives the new data is no longer dataReceived, but the lineReceived provided by LineOnlyReceiver. If you look at the Twisted source code, you can find that the interior of LineOnlyReceiver actually implements dataReceived, and then divides it by line, and calls lineReceived when there is a new line of data.

#-*-coding:utf-8-*-

From twisted.protocols.basic import LineOnlyReceiver

From twisted.internet.protocol import Factory

From twisted.internet import reactor

Class TcpServerHandle (LineOnlyReceiver):

# New connection establishment

Def connectionMade (self):

Print 'connectionMade'

# disconnect

Def connectionLost (self, reason):

Print 'connectionLost'

# received a new row of data

Def lineReceived (self, data):

Print 'lineReceived:' + data

Factory = Factory ()

Factory.protocol = TcpServerHandle

Reactor.listenTCP (8080, factory)

Reactor.run ()

Let's test three servers with a Java client.

Public class TcpClient {

Public static void main (String [] args) throws IOException {

Socket socket = null

OutputStream out = null

Try {

Socket = new Socket ("localhost", 8080)

Out = socket.getOutputStream ()

/ / request server

String lines = "there is a bright moonlight in front of the bed\ r\ n suspected frost on the ground\ r\ nlooking up at the bright moon in the sky\ r\ nlooking down and missing your hometown\ r\ n"

Byte [] outputBytes = lines.getBytes ("UTF-8")

Out.write (outputBytes)

Out.flush ()

} finally {

/ / close the connection

Out.close ()

Socket.close ()

}

}

}

MINA server output result:

SessionCreated

MessageReceived: there is bright moonlight in front of the bed

MessageReceived: like the cold frost on the ground

MessageReceived: looking up at the moon in the sky

MessageReceived: bow your head and miss your hometown

SessionClosed

Netty server output result:

ChannelActive

ChannelRead: there is bright moonlight in front of the bed

ChannelRead: like the cold frost on the ground

ChannelRead: looking up at the moon in the sky

ChannelRead: bow your head and miss your hometown

ChannelInactive

Twisted server output result:

ConnectionMade

LineReceived: there is bright moonlight in front of the bed

LineReceived: like the cold frost on the ground

LineReceived: looking up at the moon in the sky

LineReceived: bow your head and miss your hometown

ConnectionLost

Of course, during the test, you can also simulate the sent data to be segmented according to the rules, so let's test it with a more abnormal client.

Public class TcpClient {

Public static void main (String [] args) throws IOException, InterruptedException {

Socket socket = null

OutputStream out = null

Try {

Socket = new Socket ("localhost", 8080)

Out = socket.getOutputStream ()

String lines = "before bed"

Byte [] outputBytes = lines.getBytes ("UTF-8")

Out.write (outputBytes)

Out.flush ()

Thread.sleep (1000)

Lines = "bright moon"

OutputBytes = lines.getBytes ("UTF-8")

Out.write (outputBytes)

Out.flush ()

Thread.sleep (1000)

Lines = "Light\ r\ nsuspected frost on the ground\ r\ nraise your head"

OutputBytes = lines.getBytes ("UTF-8")

Out.write (outputBytes)

Out.flush ()

Thread.sleep (1000)

Lines = "looking at the bright moon\ r\ nYou look down and miss your hometown\ r\ n"

OutputBytes = lines.getBytes ("UTF-8")

Out.write (outputBytes)

Out.flush ()

} finally {

/ / close the connection

Out.close ()

Socket.close ()

}

}

}

Test the above three servers separately again, and the result is the same as the output above, without any problem.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.