Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "what is JVM direct memory". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Comparison of memory structure between JDK7 and JDK8

As can be seen from the figure above, compared with Java7, Java8 migrates the implementation of the method area from the non-heap space (in fact, the logic is connected to the heap and belongs to the runtime data area) to the local memory, which does not cause too much pressure on the FullGC and too high coupling with the old age, and reduces the scanning range of the FullGC, thus changing to the manual de-collection mechanism (which can also be automatically recycled requiring configuration adjustment). The previous article introduced the runtime data area of JVM, but there has been little research and learning on direct memory. Now we begin to introduce the allocation and recovery of direct memory.

Definition of direct memory

1. Commonly used in NIO operations, it is used for data buffers (ByteBuffer.allocate). Of course, you can also use the Unsafe class to operate the native method to apply for direct memory.

two。 The cost of allocation and recovery is high (belonging to operating system memory), but the read and write performance is high. Because direct memory does not require address mapping through the JVM interpreter to the real memory of the system, the read and write speed is much faster than that of in-heap memory, but the application and collection mechanism is complex because it is directly managed by the operating system rather than by JVM.

3. It is not managed by JVM memory collection (there is still a memory overflow problem), in addition, it is limited to the operating system's physical memory, and we can set its threshold (MaxDirectMemory) manually. By default, there is almost no limit to direct memory (when not beyond the control of physical memory).

4. The memory allocation and recycling of jvm is automatic, and there is no need to call any methods manually, but direct memory requires us to call methods manually.

Basic use of direct memory (example of IO operation)

Java heap memory:

After using direct memory, you can reduce the steps:

As you can see, direct memory does not need to be mapped and converted to java heap memory, and can go directly to operating system memory.

Memory overflow caused by direct memory

Writer: allocate direct memory each time until memory overflows

Public class Test1 {static int _ 100MB / 1024 / 1024 / 100; public static void main (String [] args) {List list=new ArrayList (); int iTuno; try {while (true) {ByteBuffer byteBuffer=ByteBuffer.allocateDirect (_ 100Mb); list.add (byteBuffer); iTunes;} finally {System.out.println (I) }}}

Test results:

Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory (Bits.java:694) at java.nio.DirectByteBuffer. (DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect (ByteBuffer.java:311) at pers.zhb.test.Test1.main (Test1.java:15) Direct memory allocation and recovery (the underlying layer is managed by Unsafe objects) Direct memory allocation and Recycling

Before running the program:

Direct memory allocation and release program:

Public class Test1 {static int _ 1Gbang 1024 1024 1024; public static void main (String [] args) throws IOException {ByteBuffer byteBuffer=ByteBuffer.allocateDirect (_ 1Gb); System.out.println ("assigned"); System.in.read (); System.out.println ("start releasing"); byteBuffer=null; System.gc ();}}

After allocating direct memory:

You can see that there is an extra java process, and it takes up 1037m of memory, so it also confirms our guess and expectation.

Click enter on the IDEA console to release the memory:

You can see that it has returned to its original state.

The console prints out tips for allocation and recycling:

After allocation, Process finished with exit code 0 is released.

Where System.gc () enforces FullGC to recycle the byteBuffer object

So far, our conjecture has all been verified, and direct memory will use another process to store the relevant system memory area, and does not belong to the management within jvm memory.

Unsafe implements the allocation and recovery of direct memory: public class Test1 {static int _ 1GB / 1024 / 1024 / 1024 / public static void main (String [] args) throws IOException {Unsafe unsafe=getUnsafe (); / / allocate memory long base=unsafe.allocateMemory (_ 1Gb); unsafe.setMemory (base,_1Gb, (byte) 0); System.in.read (); / / free memory unsafe.freeMemory (base) System.in.read ();} public static Unsafe getUnsafe () {Field field= null; try {field= Unsafe.class.getDeclaredField ("theUnsafe");} catch (NoSuchFieldException e) {e.printStackTrace ();} field.setAccessible (true); Unsafe unsafe= null Try {unsafe = (Unsafe) field.get (null);} catch (IllegalAccessException e) {e.printStackTrace ();} return unsafe;}}

The memory allocation and recycling of jvm is automatic, and there is no need to call any methods manually, but direct memory requires us to call methods manually.

ByteBuffer source code ByteBuffer (belongs to tool method, factory method)

ByteBuffer byteBuffer= ByteBuffer.allocateDirect (_ 1Gb)

AllocateDirect (memory allocation method) public static ByteBuffer allocateDirect (int capacity) {return new DirectByteBuffer (capacity);} DirectByteBuffer (object of direct memory) DirectByteBuffer (int cap) {super (- 1,0, cap, cap); boolean pa = VM.isDirectMemoryPageAligned (); int ps = Bits.pageSize (); long size = Math.max (1L, (long) cap + (pa? Ps: 0); Bits.reserveMemory (size, cap); long base = 0; try {base = unsafe.allocateMemory (size);} catch (OutOfMemoryError x) {Bits.unreserveMemory (size, cap); throw x;} unsafe.setMemory (base, size, (byte) 0) If (pa & & (base% ps! = 0)) {/ / Round up to page boundary address = base + ps-(base & (ps-1));} else {address = base;} cleaner = Cleaner.create (this, new Deallocator (base, size, cap); att = null;}

The underlying unsafe object is still used.

Advantages and disadvantages

Reduced garbage collection

Out-of-heap memory is directly managed by the operating system (not JVM). This keeps a small amount of in-heap memory to reduce the impact of garbage collection on the application.

Increase the speed of IO

The in-heap memory is managed by JVM and belongs to "user mode", while the out-of-heap memory is managed by OS and belongs to "kernel state". If you write data from the heap to disk, the data is copied to out-of-heap memory, the kernel buffer, and then written to disk by OS, which avoids this operation by using out-of-heap memory.

Shortcoming

Without JVM to help manage memory, we need to manage out-of-heap memory manually to prevent memory overflow and to avoid constant FULL GC, which eventually leads to physical memory depletion.

We will specify the maximum value of direct memory, specified by-XX:MaxDirectMemorySize, and when the threshold is reached, call system.gc to do a full gc to reclaim the unused direct memory.

In addition, the performance and speed of allocating direct memory and reclaiming memory are also complex, so the cost will be very high, especially when FullGC is needed to reclaim and allocate memory, it is necessary to switch from user mode to system mode and allocate direct memory, thus using the handle of user mode to refer to the memory space of system state.

"what is JVM direct memory" content is introduced here, thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.