How to debug the memory leak of Python programs

This article mainly explains "how to debug the memory leak of Python programs". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to debug the memory leak of Python programs.

If you run a program under Linux or macOS that may cause a memory leak, you may see something like this:

And if you are using a Windows system, then the computer may simply be stuck.

However, debugging this OOM (Out of Memory) problem is sometimes very difficult because you don't know where the code can cause a memory leak. But if you run the program to debug, the program will be killed or directly stuck in the system.

If we have a way to see the amount of memory occupied by each function in the program, then we can narrow down the scope of the check.

To do this, we can install and use a third-party library called filprofiler, which can analyze the memory footprint of Python programs.

Let's install this library first:

Pip install filprofiler

Then write a piece of code that will cause a memory leak:

Def func_a (): print ('I am a normal function') def func_b (): print ('I am the second normal function') def func_b (): print ('I am the third normal function') def func_oom (): print ('I am a function that causes a memory leak') datas = [] while True: datas.append ('s'* 1024 * 1024) print ('when running the program You won't see this line') def run (): func_a () func_b () func_oom () run ()

Running this program directly will be killed by the system directly because of the memory leak.

Before using filprofiler, you also need to resize the virtual memory. Otherwise, filprofiler itself will be killed by the system because it takes up too much memory.

First use the free command to see how much memory is available on the system:

The available memory of the system is 1619456 KB

We use the ulimit command to slightly lower the memory that the program can use, so that even if it is full, it will not be killed by the system:

Ulimit-Sv 1600000

Then, use filprofiler to run the program:

Fil-profile run test.py

The running effect is shown in the following figure:

Filprofiler will generate a fil-result folder under the current folder, with a folder named after time, and two svg files in the folder, as shown in the following figure:

When we open the out-of-memory.svg file in a browser, we can see the memory footprint shown in the following figure:

As you can see from the figure, the largest memory-consuming function is func_oom, where the program crashes.

Thank you for your reading, the above is the content of "how to debug the memory leak of Python program". After the study of this article, I believe you have a deeper understanding of how to debug the memory leak of Python program. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!