Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you an example analysis of NIO and zero copy. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

What is NIO

Let's take a look at the following code:

File file = new File ("test.txt")

RandomAccessFile raf = new RandomAccessFile (file, "rw")

Byte [] arr = new byte [(int) file.length ()]

Raf.read (arr)

Socket socket = new ServerSocket (8888) .accept ()

Socket.getOutputStream () write (arr)

This code is to read a file, and then write it out, it looks like a few lines of code, but actually involves multiple copies, the process is as follows:

There are two steps in the process of reading the data to be copied: first, the file is copied to the buffer of the system kernel through DMA (direct memory access, direct memory copy, without CPU), and then it is copied to the user's buffer through CPU in the kernel buffer. That is, two copies have taken place in the first four lines.

When writing data, the data is copied from the user buffer to socket buffer through CPU, and finally from socket buffer to the protocol stack through DMA. Even the last line has been copied twice.

During the whole process, there were four copies and three state transitions. From the beginning of the user state, switch to the kernel state, then to the user state, and finally to the kernel state. A simple read and write, there are so many things, the performance is definitely not good, all there is zero copy, zero copy, not no copy, but the whole process does not need to be CPU copy.

Second, zero copy

1. Use mmap to optimize the above process: mmap means that files are mapped to kernel buffers through memory mapping. at the same time, user space can share the data in kernel space, so that the number of copies from kernel space to user space can be reduced during network transfer. To do the same thing above, the whole process when using mmap is as follows:

First, copy the hard disk data to the kernel buffer through the DMA copy, but because the user buffer can share the kernel buffer data, the cpu copy in step 2 is avoided.

Then it is copied directly from the kernel buffer to socket buffer via CPU, and finally DMA to the protocol stack.

The whole process of three copies, three state changes, compared with the traditional copy, optimize a lost, but this is not zero copy.

2. Optimize with sendFile: sendFile:sendFile of linux2.1 is a function provided by linux2.1 version, which allows files to enter socket buffer directly from kernel buffer without going through user mode. The process is as follows:

First, copy the data from the hard disk to the kernel buffer through DMA, then copy the data from the kernel buffer to the socket buffer through the CPU copy, and finally copy the data to the protocol stack through DMA.

The whole process is still three copies, but one switch is reduced, from the user mode to the kernel state and then to the user mode, there are only two switches. There is still one copy of CPU here, which is not a real zero copy.

SendFile:linux 2.4 of linux 2.4 makes some optimizations for sendFile, first DMA copy to kernel buffer, then CPU copy to socket buffer, and finally DMA copy to the protocol stack. The point of optimization is that the content of this CPU copy is very small, only the length and offset of the kernel buffer are copied, and the consumption is very low and can be ignored. So, this is zero copy. The transferTo method of NIO can achieve zero copy.

III. Case code

1. Traditional IO copies large files:

Server: receives the data sent by the client public class OldIoServer {

@ SuppressWarnings ("resource")

Public static void main (String [] args) throws IOException {

ServerSocket serverSocket = new ServerSocket (6666)

While (true) {

Socket socket = serverSocket.accept ()

DataInputStream dataInputStream = new DataInputStream (socket.getInputStream ())

Byte [] byteArray = new byte [4096]

While (true) {

Int readCount = dataInputStream.read (byteArray, 0, byteArray.length)

If (- 1 = = readCount) {

Break

}

}

}

}

}

Client: send a large file to the server: public class OldIoClient {

@ SuppressWarnings ("resource")

Public static void main (String [] args) throws Exception {

Socket socket = new Socket ("127.0.0.1", 6666)

/ / Files to be copied

String fileName = "E:\ download\\ soft\\ windows\\ jdk-8u171-windows-x64.exe"

InputStream inputStream = new FileInputStream (fileName)

DataOutputStream dataOutputStream = new DataOutputStream (socket.getOutputStream ())

Byte [] buffer = new byte [4096]

Long readCount

Long total = 0

Long start = System.currentTimeMillis ()

While ((readCount = inputStream.read (buffer)) > = 0) {

Total + = readCount

DataOutputStream.write (buffer)

}

Long end = System.currentTimeMillis ()

System.out.println ("Total bytes transferred:" + total + ", time:" + (end-start) + "milliseconds")

DataOutputStream.close ()

InputStream.close ()

Socket.close ()

}

}

A JDK is copied here, and the final running result is as follows:

Total bytes transferred: 217342912, time: 4803 milliseconds

As you can see, for nearly five seconds. Let's take a look at the time consuming using the transferTo method of NIO:

Server: public class NioServer {

Public static void main (String [] args) throws IOException {

InetSocketAddress address = new InetSocketAddress (6666)

ServerSocketChannel serverSocketChannel = ServerSocketChannel.open ()

ServerSocket serverSocket = serverSocketChannel.socket ()

ServerSocket.bind (address)

ByteBuffer buffer = ByteBuffer.allocate (4096)

While (true) {

SocketChannel socketChannel = serverSocketChannel.accept ()

Int readCount = 0

While (- 1! = readCount) {

ReadCount = socketChannel.read (buffer)

Buffer.rewind (); / / rewind, set position to 0 and mark to-1

}

}

}

}

Client: public class NioClient {

@ SuppressWarnings ("resource")

Public static void main (String [] args) throws IOException {

SocketChannel socketChannel = SocketChannel.open ()

SocketChannel.connect (new InetSocketAddress ("127.0.0.1", 6666))

String fileName = "E:\ download\\ soft\\ windows\\ jdk-8u171-windows-x64.exe"

FileChannel channel = new FileInputStream (fileName). GetChannel ()

Long start = System.currentTimeMillis ()

/ / under linux, the transferTo method can send data at one time

/ / in windows, the file transferred by transferTo method is segmented if it exceeds 8m.

Long totalSize = channel.size ()

Long transferTotal = 0

Long position = 0

Long count = 8 * 1024 * 1024

If (totalSize > count) {

BigDecimal totalCount = new BigDecimal (totalSize) .setting (new BigDecimal (count)) .setScale (0, RoundingMode.UP)

For (int iTunes 1; I

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.