Shulou(Shulou.com)06/03 Report--

This article mainly introduces the meaning of memory leak and memory overflow in java. It is very detailed and has certain reference value. Friends who are interested must finish reading it.

A memory leak means that the program is unable to release the requested memory space after requesting memory. Memory overflow means that when the program applies for memory, there is not enough memory for the applicant to use; or to provide a piece of storage space to store int data, but to store long data, the result is that there is not enough memory and an error OOM is reported. The accumulation of memory leaks will eventually lead to memory overflows.

The operating environment of this tutorial: windows7 system, java8 version, DELL G3 computer.

1. Memory leak memory leak:

It means that the program cannot release the applied memory space after applying for memory, and a memory leak does not seem to have a big impact, but the consequence of memory leak accumulation is memory overflow.

2. Memory overflow out of memory:

It means that when the program applies for memory, there is not enough memory for the applicant to use, that is to say, it gives you a piece of storage space to store int type data, but you store long type data, then the result is that there is not enough memory, and an error OOM will be reported at this time, that is, the so-called memory overflow.

3. The relationship between the two:

The accumulation of memory leaks will eventually lead to memory overflow.

Memory overflow means that the memory space you want exceeds the space actually allocated to you by the system. When the system is unable to meet your needs, it will report a memory overflow error.

Memory leak means that you apply to the system for allocating memory for use (new), but do not return it after use (delete). As a result, you can no longer access the piece of memory you applied for (maybe you lost its address), and the system cannot allocate it to the needed program again. It is equivalent to renting a cabinet with a key. after you lock the cabinet after you have saved something, you lose the key or do not return the key, then the result is that the cabinet will not be available to anyone and cannot be recycled by the garbage collector. because I can't find any information about him.

Memory overflow: a plate can only hold 4 fruits in various ways, but you have 5, and you fall to the ground and can't eat it. This is spillover. For example, the stack, when the stack is full, it is bound to produce space overflow, which is called upper overflow, and when the stack is empty, it also produces space overflow, which is called underflow. That is, the allocated memory is not enough to drop the sequence of data items, which is called memory overflow. To put it bluntly, I can't bear so much, so I'm wrong.

4. Classification of memory leaks (classified by the way they occur)

Frequent memory leaks. Code that has a memory leak is executed multiple times, resulting in a memory leak each time it is executed.

Occasional memory leak. Code that has a memory leak occurs only in certain environments or operations. Recurrent and sporadic are relative. For a particular environment, sporadic may become frequent. Therefore, the test environment and test methods are very important to detect memory leaks.

One-time memory leak. The code that has a memory leak will only be executed once, or because of an algorithmic flaw, there will always be a leak of only one piece of memory. For example, memory is allocated in the constructor of a class, but not freed in the destructor, so a memory leak occurs only once.

Implicit memory leak. The program keeps allocating memory while it is running, but it doesn't release memory until the end. Strictly speaking, there was no memory leak here, because in the end, the program freed all the requested memory. But for a server program, it takes days, weeks, or even months to run, and not releasing memory in time can eventually deplete all the memory in the system. Therefore, we call this type of memory leak an implicit memory leak.

5. The causes and solutions of memory overflow:

(1) the cause of memory overflow:

The amount of data loaded in memory is too large, such as fetching too much data from the database at once

There is a reference to the object in the collection class, which is not cleared after use, so that the JVM cannot be recycled.

There is an endless loop in the code or an object entity whose loop produces too much repetition.

BUG in the third-party software used

The memory value of startup parameters is set too small.

(2) solution to memory overflow:

The first step is to modify the JVM startup parameters to directly increase memory. (don't forget to add the parameter-Xms,-Xmx.)

The second step is to check the error log to see if there are other exceptions or errors before the "OutOfMemory" error.

The third step is to walk through and analyze the code to find out where the memory overflow may occur.

Focus on the following points:

Check whether there is a query to get all the data in the database query. Generally speaking, if 100, 000 records are taken to memory at a time, it may cause a memory overflow. This problem is more hidden, before going online, there is less data in the database, so it is not easy to cause problems. After going online, there is more data in the database, and a query may cause memory overflow. Therefore, the database query should be paged as far as possible.

Check the code for endless loops or recursive calls.

Check to see if there are large loops that repeatedly generate new object entities.

Check whether there is a query to get all the data in the database query. Generally speaking, if 100, 000 records are taken to memory at a time, it may cause a memory overflow. This problem is more hidden, before going online, there is less data in the database, so it is not easy to cause problems. After going online, there is more data in the database, and a query may cause memory overflow. Therefore, the database query should be paged as far as possible.

Check whether there are any problems that have not been cleared after the collection objects such as List and MAP are finished. Collection objects such as List, MAP, etc., always have references to objects, so that these objects cannot be recycled by GC.

The fourth step is to use the memory View tool to dynamically view memory usage

JVM8 memory model

Ten scenarios of memory overflow

The JVM runtime first needs the class loader (classLoader) to load the bytecode file of the desired class. After loading, it is left to the execution engine, and a section of space is needed to store data during execution (analogy CPU to main memory). This process of allocating and freeing memory space is exactly the runtime data area that we need to care about. Memory overflows occur when the classloader loads, and memory overflows fall into two main categories: OutOfMemoryError and StackOverflowError. Here are 10 cases of memory overflow, and explain how the memory overflow occurs in the way of example code.

1.java heap memory overflow

When a java.lang.OutOfMemoryError:Java heap space exception occurs, the heap memory is overflowed.

1), problem description

The jvm memory set is too small, the memory required for the object is too large, and this exception will be thrown if space is allocated when the object is created.

At the peak of traffic / data, there is a limit to the processing of the application itself, such as a certain number of users or a certain amount of data. When the number of users or data surges suddenly and exceeds the expected threshold, then the normal operation before the peak stops will stop and trigger java. Lang.OutOfMemoryError:Java heap space error

2), sample code

Compile the following code with the jvm parameter set to-Xms20m-Xmx20m when executed

In the above example, if only 5m of memory is allocated for one request, garbage collection will not go wrong if the number of requests is small, but once the concurrency exceeds the maximum memory value, a memory overflow will be thrown.

3. Solution method

First of all, if there is nothing wrong with the code, you can adjust the-Xms and-Xmx jvm parameters appropriately, and use stress tests to adjust them to the optimal value.

Secondly, try to avoid large object applications, such as file upload, get large quantities from the database, which needs to be avoided, as far as possible block or batch processing, which is conducive to the normal and stable implementation of the system.

Finally, try to improve the execution speed of a request, garbage collection as early as possible, otherwise, when a large number of concurrency comes, new requests will not be able to allocate memory, which will easily cause an avalanche of the system.

2. Java heap memory leak

1), problem description

A memory leak in Java is a situation in which objects are no longer used by applications but are not recognized by garbage collection. Therefore, these unused objects still exist indefinitely in the Java heap space. Constant accumulation will eventually trigger java. Lang.OutOfMemoryError .

2), sample code

When executing the above code, you might expect it to run forever without any problems, assuming that the simple caching solution only extends the underlying mapping to 10000 elements, not all the keys are already in the HashMap. In fact, however, the element will continue to be added because the key class does not override its equals () method.

Over time, with leaked code in use, the result of "caching" will eventually consume a lot of Java heap space. Garbage collection cannot clean up memory when leaking all available memory in the heap area, java. Lang.OutOfMemoryError .

3) solution

The corresponding solution is relatively simple: override the equals method:

3. Garbage collection timeout memory overflow

1) problem description when the application uses up all available memory, the GC overhead limit exceeds the error, and GC fails to clear it many times, then a java.lang.OutOfMemoryError is raised. When JVM spends a lot of time executing GC, with little effect, an error will be triggered once the entire GC process exceeds the limit (by default, jvm configures GC for more than 98% and reclaims heap memory less than 2%).

2), sample code

3), solution

To reduce the life cycle of objects, garbage collection can be carried out as quickly as possible.

4.Metaspace memory overflow

1), problem description

If the metaspace overflows, the system throws java.lang.OutOfMemoryError: Metaspace. The reason for this exception is that the system has a lot of code or refers to a lot of third-party packages or uses methods such as dynamic code generation class loading, which results in a large amount of meta-space.

2), sample code

3) solution

By default, the size of metaspace is limited only by local memory. However, for the performance of the whole machine, it is necessary to set this item as far as possible, so as not to cause the service of the whole machine to stop.

Optimize parameter configuration to avoid affecting other JVM processes

-XX:MetaspaceSize, the initial space size, which triggers garbage collection to unload types, and GC adjusts the value: if a large amount of space is freed, the value is lowered appropriately; if little space is freed, the value is increased appropriately when the MaxMetaspaceSize is not exceeded.

-XX:MaxMetaspaceSize, the maximum space, there is no limit by default.

In addition to the above two specified size options, there are two GC-related properties:-XX:MinMetaspaceFreeRatio, after GC, the minimum percentage of Metaspace remaining space capacity, reducing garbage collection caused by space allocation. -XX:MaxMetaspaceFreeRatio, the percentage of the maximum Metaspace remaining space capacity after GC, reducing garbage collection caused by freeing space.

Carefully quote third-party packages

For third-party bags, be sure to choose carefully and remove those you don't need. This not only helps improve the speed of compilation and packaging, but also helps improve the speed of remote deployment.

Focus on the framework of dynamically generated classes

For frameworks that use a large number of dynamically generated classes, stress tests should be done to verify that dynamically generated classes exceed memory requirements and throw exceptions.

5. Direct memory overflow

1), problem description

When using allocateDirect () in ByteBuffer, many javaNIO (like netty) are encapsulated as other methods in the framework, and the java.lang.OutOfMemoryError: Direct buffer memory exception is thrown when this problem occurs.

A similar problem occurs if you use the allocateDirect method in ByteBuffer directly or indirectly, instead of doing clear.

2), sample code

3) solution

If you often have similar operations, consider setting the parameter:-XX:MaxDirectMemorySize, and clear memory in time.

6. Stack memory overflow

1), problem description

When a thread executes a Java method, JVM creates a new stack frame and push it to the top of the stack. At this point, the new stack frame becomes the current stack frame, and when the method is executed, the stack frame is used to store parameters, local variables, intermediate instructions, and other data.

When a method calls itself recursively, the data generated by the new method (which can also be understood as the new stack frame) will be push to the top of the stack. Each time the method calls itself, it will copy a copy of the current method's data and push it to the stack. Therefore, each layer of recursive call needs to create a new stack frame. As a result, more and more memory in the stack will be consumed with recursive calls, and if you call yourself a million times recursively, a million stack frames will be generated. This will cause a memory overflow of the stack.

2), sample code

3) solution

If there is a recursive call in the program, when there is a stack overflow, you can increase the-Xss size to solve the stack memory overflow problem. Recursive calls prevent the formation of a dead loop, otherwise stack memory overflows will occur.

7. Create a local thread memory overflow

1), problem description

The thread basically only occupies the memory area outside the heap, that is, this error indicates that it is impossible to allocate a memory area for the thread except the heap area, either the memory itself is not enough, or the heap space is set too large, resulting in not much memory left, and because the thread itself has to occupy memory, it is not enough.

2), sample code

3), solution

First check whether the operating system has a limit on the number of threads, and you cannot create threads with shell. If this is the problem, you need to adjust the maximum number of files that the system can support.

Try to keep the maximum number of threads under control in daily development, and do not use thread pools at will. It cannot continue to grow indefinitely.

8. Memory overflow out of swap area

1), problem description

During the startup of a Java application, you can limit the specified required memory with-Xmx and other similar startup parameters. When the total memory requested by JVM is greater than the available physical memory, the operating system begins to convert the content from memory to hard disk.

Generally speaking, JVM will throw an Out of swap space error, which means that when the application fails to allocate memory to JVM native heap and the native heap is about to be exhausted, the error message contains the size of the allocation failure (in bytes) and the reason for the request failure.

2) solution

To increase the size of the swap area of the system, I personally think that if the swap area is used, the performance will be greatly reduced, which is not recommended. The production environment should try to avoid the maximum memory exceeding the physical memory of the system. Secondly, get rid of the system switching area and only use the system memory to ensure the performance of the application.

9. Array out of limit memory overflow

1), the problem description will sometimes encounter this kind of memory overflow description Requested array size exceeds VM limit. Generally speaking, java has a limit on the maximum size of the array that the application can allocate, but the restrictions vary from platform to platform, but it is usually between 1 and 2.1 billion elements. When a Requested array size exceeds VM limit error occurs, it means that the application is trying to allocate an array larger than the Java virtual machine can support. Before allocating memory for an array, JVM performs a platform-specific check to see if the allocated data structure is addressable on that platform.

2), sample code

The following is the code that the array exceeds the maximum limit.

3), solution

Therefore, the array length should be within the length allowed by the platform. However, this error is generally rare, mainly because the index of the Java array is of type int. The largest positive integer in Java is 2 ^ 31-1 = 2147483647. And platform-specific restrictions can be very close to this number, for example: my environment (64-bit macOS, running Jdk1.8) can initialize arrays as long as 2147483645 (Integer.MAX_VALUE-2). If you increase the length of the array by another 1 to reach the OutOfMemoryError that nteger.MAX_VALUE-1 will appear.

10. The system kills the process memory overflow

1) problem Overview before describing the problem, familiarize yourself with a little bit of operating system knowledge: the operating system is based on the concept of processes that work in the kernel, including a very special process called "memory killer (Out of memory killer)". When the kernel detects that the system is out of memory, OOM killer is activated to check who currently occupies the most memory and kill the process.

A general Out of memory:Kill process or sacrifice child error is triggered when available virtual virtual memory (including swap space) is consumed to put the entire operating system at risk. In this case, OOM Killer will choose the "rogue process" and kill it.

2), sample code

3), solution

Although increasing swap space can alleviate Java heap space exceptions, it is recommended that the best solution is to upgrade system memory so that enough memory is available for java applications, so that this problem will not occur.

Summing up the above 10 memory overflow situations, you will know how to solve the problem when you actually encounter a problem, and you should also remember in the actual coding:

Third-party jar packages should be introduced cautiously, resolutely get rid of useless jar packages, and improve the speed of compilation and the memory occupied by the system.

For large objects or a large number of memory applications, to optimize, large objects should be segmented to improve processing performance and reduce the object life cycle.

Try to fix the number of threads to ensure that the memory occupied by threads can be controlled, and when a large number of threads are needed, the maximum number of connections that can be opened by the operating system should be optimized.

For recursive calls, you should also control the level of recursion, not too high, beyond the depth of the stack.

The larger the memory allocated to the stack, the better, because the larger the stack memory, the more threads, leaving less space for the heap, making it easy to throw OOM. The default parameters of JVM are generally fine (including recursion).

These are all the contents of the article "what do memory leaks and memory overflows in java mean?" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.