How to limit the memory that your Python program can use
This article mainly explains "how to limit the memory that your Python program can use". The explanation in this article is simple and clear, easy to learn and understand. Please follow the ideas of Xiaobian and study and learn "how to limit the memory that your Python program can use" together!
If the program is not developed properly, it may take up too much memory. Especially in Docker, if Python programs consume too much memory, Docker containers may die.
To limit the maximum memory Python programs can use, we can use Python's own resource module.
First get the system default soft and hard memory caps:
import resource soft, hard = resource.getrlimit(resource.RLIMIT_AS)
The resource.RLIMIT_AS is actually the number 5, representing memory resources. A soft limit is an adjustable upper limit on memory usage, and a hard limit is an upper limit on memory usage that is difficult to dynamically adjust once set.
Running it on my computer looks like this:
The soft and hard values are the same at first. And their units are not bits. but units smaller than bits. My computer has 8GB of RAM, and 8 * 1024 ^ 6 is exactly equal to the hard and soft values here.
Next, set the maximum memory that the currently running Python program can use:
resource.setrlimit(resource.RLIMIT_AS, (max upper limit, hard))
where the maximum upper bound is an integer. Suppose I want to limit the current program to 500MB of memory, then the maximum limit here can be set to:
0.5 * 1024 ^ 6 = 576460752303423488
So set it to:
resource.setrlimit(resource.RLIMIT_AS, (576460752303423488, hard))
When the current Python program uses more than 500MB of memory, the program will throw MemoryError.
Finally, this code only works properly on Linux systems. There may be problems on Windows and Mac.
Thank you for reading, the above is the content of "how to limit the memory that your Python program can use", after learning this article, I believe everyone has a deeper understanding of how to limit the memory that your Python program can use, and the specific use needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!
