Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "how to understand the zero-copy technology in Netty and Kafka". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Buffer zone

The buffer is the basis of all IWeiO, which is nothing more than moving data into or out of the buffer; when a process performs IWeiO, it sends a request to the operating system to either drain (write) or fill the buffer (read).

Let's take a look at a general flow chart of a Java process initiating a Read request to load data:

After the process initiates the Read request, after receiving the Read request, the kernel will first check whether the data needed by the process already exists in the kernel space, and if so, Copy the data directly to the buffer of the process.

If no kernel immediately issues a command to the disk controller to read data from the disk, the disk controller writes the data directly to the kernel Read buffer, which is done through DMA.

Next, the kernel Copy the data to the process's buffer; if the process initiates a Write request, it also needs to Copy the data in the user buffer to the kernel's Socket buffer, and then Copy the data to the network card through DMA and send it out.

You may think this is a waste of space, and you need to copy the data from kernel space to user space every time, so zero copy appears to solve this problem.

There are two ways to provide zero copy:

Mmap+write

Sendfile

Virtual memory

All modern operating systems use virtual memory and use virtual addresses instead of physical addresses, which has the advantage of:

More than one virtual address can point to the same physical memory address.

The virtual memory space can be larger than the actual available physical address.

With the first feature, kernel space addresses and user space virtual addresses can be mapped to the same physical address, so that DMA can fill buffers that are visible to both kernel and user space processes.

Roughly as shown in the following figure:

Omitting the copy between kernel and user space, Java also takes advantage of this feature of the operating system to improve performance. Let's focus on what Java supports for zero copy.

Mmap+write mode

Using mmap+write instead of the original read+write, mmap is a method of memory mapping files, that is, a file or other objects are mapped to the address space of the process to realize the one-to-one correspondence between the file disk address and a virtual address in the process virtual address space.

This saves the original kernel Read buffer Copy data to the user buffer, but still requires the kernel Read buffer to Copy the data to the kernel Socket buffer.

Roughly as shown in the following figure:

Sendfile mode

Sendfile system calls were introduced in kernel version 2.1 to simplify the process of data transfer between two channels over the network.

The introduction of Sendfile system call not only reduces data replication, but also reduces the number of context switching, as shown in the following figure:

Data transfer only occurs in kernel space, so one context switch is reduced; but there is still a Copy, can you omit the Copy this time?

Improvements have been made in the Linux2.4 kernel to record the corresponding data description information (memory address, offset) in the Kernel buffer into the corresponding Socket buffer, thus eliminating a CPU Copy in the kernel space.

Java zero copy

MappedByteBuffer

The FileChannel provided by Java NIO provides the map () method, which establishes a virtual memory map between an open file and MappedByteBuffer.

MappedByteBuffer inherits from ByteBuffer and is similar to a memory-based buffer, except that the data elements of the object are stored in a file on disk.

Calling the get () method fetches the data from disk, which reflects the current contents of the file, calling the put () method updates the file on disk, and the changes made to the file are visible to other readers.

Let's look at a simple read example, and then analyze the MappedByteBuffer:

Public class MappedByteBufferTest {public static void main (String [] args) throws Exception {File file = new File ("D://db.txt"); long len = file.length (); byte [] ds = new byte [(int) len]; MappedByteBuffer mappedByteBuffer = new FileInputStream (file). GetChannel (). Map (FileChannel.MapMode.READ_ONLY, 0, len); for (int offset = 0; offset < len) Offset++) {byte b = mappedByteBuffer.get (); ds [offset] = b;} Scanner scan = new Scanner (new ByteArrayInputStream (ds)). UseDelimiter (""); while (scan.hasNext ()) {System.out.print (scan.next () + ");}

The mapping is mainly implemented through map () provided by FileChannel, and the map () method is as follows:

Public abstract MappedByteBuffer map (MapMode mode, long position, long size) throws IOException

Three parameters, MapMode,Position and Size, are provided to indicate:

MapMode: mode of mapping. Options include: READ_ONLY,READ_WRITE,PRIVATE.

Position: where to start the mapping and the location of the number of bytes.

Size: how many bytes backwards from Position.

Let's focus on MapMode. The first two represent read-only and read-write, respectively. Of course, the requested mapping mode is restricted by the access permission of the Filechannel object. If READ_ONLY is enabled on a file that does not have read permission, NonReadableChannelException will be thrown.

The PRIVATE schema represents the mapping of the copy at write time, meaning that any changes made through the put () method result in a private copy of the data in which only the MappedByteBuffer instance can see it.

This process does not make any changes to the underlying files, and once the buffer is garbage collected, those changes are lost.

Take a quick look at the source code of the map () method:

Public MappedByteBuffer map (MapMode mode, long position, long size) throws IOException {... Omit. Int pagePosition = (int) (position% allocationGranularity); long mapPosition = position-pagePosition; long mapSize = size + pagePosition; try {/ / If no exception was thrown from map0, the address is valid addr = map0 (imode, mapPosition, mapSize) } catch (OutOfMemoryError x) {/ / An OutOfMemoryError may indicate that we've exhausted memory / / so force gc and re-attempt map System.gc (); try {Thread.sleep (100);} catch (InterruptedException y) {Thread.currentThread () .interrupt () } try {addr = map0 (imode, mapPosition, mapSize);} catch (OutOfMemoryError y) {/ / After a second OOME, fail throw new IOException ("Map failed", y) }} / / On Windows, and potentially other platforms, we need an open / / file descriptor for some mapping operations. FileDescriptor mfd; try {mfd = nd.duplicateForMapping (fd);} catch (IOException ioe) {unmap0 (addr, mapSize); throw ioe;} assert (IOStatus.checkAll (addr)); assert (addr% allocationGranularity = = 0); int isize = (int) size Unmapper um = new Unmapper (addr, mapSize, isize, mfd) If ((! writable) | | (imode = = MAP_RO) {return Util.newMappedByteBufferR (isize, addr + pagePosition, mfd, um) } else {return Util.newMappedByteBuffer (isize, addr + pagePosition, mfd, um);}}

It roughly means that the address of the memory map is obtained by the Native method, and if it fails, the manual GC is mapped again.

Finally, the memory-mapped address instantiates that MappedByteBuffer,MappedByteBuffer itself is an abstract class, but what is really instantiated here is DirectByteBuffer.

DirectByteBuffer

DirectByteBuffer inherits from MappedByteBuffer, and you can guess from the name that it opens up a period of direct memory and does not take up the memory space of JVM.

The MappedByteBuffer mapped through Filechannel in the previous section is actually DirectByteBuffer. Of course, in addition to this way, you can also manually open up a space:

ByteBuffer directByteBuffer = ByteBuffer.allocateDirect

This opens up 100 bytes of direct memory space.

Channel-to-Channel transmission

You often need to transfer files from one location to another. FileChannel provides a transferTo () method to improve the efficiency of the transfer. First, look at a simple example:

Public class ChannelTransfer {public static void main (String [] argv) throws Exception {String files [] = new String [1]; files [0] = "D://db.txt"; catFiles (Channels.newChannel (System.out), files);} private static void catFiles (WritableByteChannel target, String [] files) throws Exception {for (int I = 0; I < files.length) FileInputStream fis +) {FileInputStream fis = new FileInputStream (Files [I]); FileChannel channel = fis.getChannel (); channel.transferTo (0, channel.size (), target); channel.close (); fis.close ();}

The file data is transferred to the System.out channel through the transferTo () method of FileChannel. The API is defined as follows:

Public abstract long transferTo (long position, long count, WritableByteChannel target) throws IOException

Several parameters are also easy to understand: where to start the transfer, the number of bytes transferred, and the target channel; transferTo () allows you to cross-connect one channel to another without the need for an intermediate buffer to pass data.

Note: there is no need for an intermediate buffer that has two meanings: the first layer does not need a user space buffer to copy the kernel buffer, and the other layer has its own kernel buffer for both channels, and the two kernel buffers do not need to copy data.

Netty zero copy

Netty provides zero-copy Buffer. When transmitting data, the final processed data will need to combine and split a single transmitted message, which can not be achieved by NIO native ByteBuffer. Netty achieves zero copy by providing Composite (combination) and Slice (split) Buffer.

It will be clearer if you look at the following picture:

The HTTP message in the TCP layer is divided into two ChannelBuffer, which are meaningless to our upper logic (HTTP processing).

But when the two ChannelBuffer are combined, they become a meaningful HTTP message, and the corresponding ChannelBuffer of this message is what can be called "Message". Here the word "Virtual Buffer" is used.

You can take a look at the CompositeChannelBuffer source code provided by Netty:

Public class CompositeChannelBuffer extends AbstractChannelBuffer {private final ByteOrder order; private ChannelBuffer [] components; private int [] indices; private int lastAccessedComponentId; private final boolean gathering; public byte getByte (int index) {int componentId = componentId (index); return principals [implementId] .getByte (index-indices [componentId]);}. Omit.

What Components uses to save is that all received Buffer,Indices records the starting position of each buffer, and lastAccessedComponentId records the last visited ComponentId.

Instead of opening up new memory and copying all ChannelBuffer content directly, CompositeChannelBuffer directly saves all references to ChannelBuffer and reads and writes in a child ChannelBuffer, achieving zero copy.

Other zero copy

RocketMQ's messages are written sequentially to the commitlog file, and then the consume queue file is used as the index.

RocketMQ responds to Consumer requests with zero-copy mmap+write.

Similarly, there are a large number of network data persistence to disk and disk files sent through the network in Kafka, and Kafka uses Sendfile zero-copy mode.

This is the end of the content of "how to understand the zero-copy technology in Netty and Kafka". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.