Shulou(Shulou.com)06/02 Report--

This article mainly introduces "Python memory management example analysis". In daily operation, I believe many people have doubts about Python memory management example analysis problems. The editor consulted all kinds of data and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "Python memory management example analysis". Next, please follow the editor to study!

Language memory management is an important aspect of language design. It is an important factor in determining language performance. Both manual management of C language and garbage collection of Java have become the most important features of the language. Here we take Python as an example to illustrate the memory management of a dynamically typed, object-oriented language.

Memory usage of object

Assignment statements are the most common function of a language. But even the simplest assignment statement can have a lot of meaning. The assignment statement of Python is worth studying.

A = 1

The integer 1 is an object. And an is a reference. Using the assignment statement, the reference a points to object 1. Python is a dynamically typed language (refer to dynamically typed), and objects are separated from references. Like using chopsticks, Python uses references to touch and flip the real food-the object.

References and objects

To explore how objects are stored in memory, we can turn to Python's built-in function id (). It is used to return the object's identity (identity). In fact, the so-called identity here is the memory address of the object.

A = 1 print (id (a)) print (hex (id (a)

On my computer, they return:

11246696 '0xab9c68'

The decimal and hexadecimal representations of the memory address, respectively.

In Python, both integers and short characters are cached by Python for reuse. When we create multiple references equal to 1, we actually have all of these references point to the same object.

A = 1 b = 1 print (id (a)) print (id (b))

The above program returns

11246696 11246696

You can see that an and b are actually two references to the same object.

To verify that two references point to the same object, we can use the is keyword. Is is used to determine whether the objects referred to by two references are the same.

# Truea = 1b = 1 print (an is b) # Truea = "good" b = "good" print (an is b) # False a = "very good morning" b = "very good morning" print (an is b) # False a = [] b = [] print (an is b)

The comments above are the corresponding running results. As you can see, because Python caches integers and short strings, there is only one copy per object. For example, all references to integer 1 point to the same object. Even if you use assignment statements, you only create new references, not the object itself. Long strings and other objects can have multiple identical objects, and new objects can be created using assignment statements.

In Python, each object has the total number of references to that object, that is, the reference count (reference count).

We can use getrefcount () in the sys package to see the reference count of an object. It is important to note that when a reference is passed to getrefcount () as a parameter, the parameter actually creates a temporary reference. Therefore, getrefcount () will get 1 more results than expected.

From sys import getrefcount a = [1,2,3] print (getrefcount (a)) b = a print (getrefcount (b))

For the above reasons, the two getrefcount will return 2 and 3 instead of the expected 1 and 2.

Object reference object

A container object (container) of Python, such as tables, dictionaries, and so on, can contain multiple objects. In fact, what the container object contains is not the element object itself, but a reference to each element object.

We can also customize one object and reference other objects:

Class from_obj (object): def _ init__ (self, to_obj): self.to_obj = to_obj b = [1Mei 2jue 3] a = from_obj (b) print (id (a.to_obj)) print (id (b))

As you can see, a references object b.

Object refers to object, which is the most basic way of constructing Python. Even the assignment of a = 1 actually allows an element of the dictionary with the key value "a" to refer to the integer object 1. This dictionary object is used to record all global references. The dictionary refers to the integer object 1. We can view the dictionary through the built-in function globals ().

When one object An is referenced by another object B, the reference count of An increases by 1.

From sys import getrefcount a = [1,2,3] print (getrefcount (a)) b = [a, a] print (getrefcount (a))

The reference count of aMagi an increased by 2 because object b referenced twice.

References to container objects can form a complex topology. We can use the objgraph package to draw its reference relationships, such as

X = [1,2,3] y = [x, dict (key1=x)] z = [y, (x, y)] import objgraph objgraph.show_refs ([z], filename='ref_topo.png')

Objgraph is a third-party package for Python. Xdot needs to be installed before installation.

Sudo apt-get install xdot sudo pip install objgraph

Objgraph official website

Two objects may refer to each other to form a so-called reference cycle.

A = [] b = [a] a.append (b)

Even an object that only needs to quote itself can form a reference ring.

A = [] a.append (a) print (getrefcount (a))

The reference ring can cause a lot of trouble for the garbage collection mechanism, which I will describe in more detail later.

Citation reduction

The reference count of an object may be reduced. For example, you can use the del keyword to delete a reference:

From sys import getrefcount a = [1,2,3] b = a print (getrefcount (b)) del a print (getrefcount (b))

Del can also be used to delete elements in container elements, such as:

Del a [0] print (a)

If a reference points to object A, when the reference is redirected to some other object B, the reference count of object A decreases:

From sys import getrefcount a = [1,2,3] b = a print (getrefcount (b)) a = 1 print (getrefcount (b)) garbage collection

If you eat too much, you will always get fat, and so will Python. As there are more and more objects in the Python, they will occupy more and more memory. But you don't have to worry too much about Python's figure, it will cleverly "lose weight" at the right time, start garbage collection (garbage collection), and get rid of useless objects. Garbage collection mechanisms are available in many languages, such as Java and Ruby. Although the ultimate goal is to make slim reminders, weight loss programs vary greatly from language to language (this can be compared with Java memory management and garbage collection

).

Basically, when the reference count of an object in Python drops to 0, it means that there are no references to the object, and the object becomes garbage to be recycled. For example, a new object is assigned to a reference, and the reference count of the object becomes 1. If the reference is deleted and the reference count of the object is 0, the object can be garbage collected. For example, the following table:

A = [1,2,3] del a

After del a, there are no references to the previously created [1, 2, 3] table. It is impossible for the user to touch or use this object in any way. If the object stays in memory, it will become unhealthy fat. When garbage collection starts, Python scans the object with a reference count of 0 and empties its memory.

However, losing weight is an expensive and laborious thing. Python cannot perform other tasks during garbage collection. Frequent garbage collection will greatly reduce the efficiency of Python. If there are not many objects in memory, there is no need to always start garbage collection. Therefore, Python will only automatically start garbage collection under certain conditions. When Python runs, the number of times the allocated object (object allocation) and the unallocated object (object deallocation) are recorded. Garbage collection starts when the difference between the two is higher than a certain threshold.

We can view this threshold through the get_threshold () method of the gc module:

Import gc print (gc.get_threshold ())

Return (700, 10, 10), and the next two 10s are the thresholds related to generational recycling, as you can see later. 700 is the threshold for garbage collection to start. It can be reset through the set_threshold () method in gc.

We can also start garbage collection manually, using gc.collect ().

Generation by generation recovery

Python also adopts the strategy of generation recycling. The basic assumption of this strategy is that the longer the object lives, the less likely it is to become garbage in later programs. Our programs tend to produce a large number of objects, many of which are quickly generated and disappear, but some objects are used for a long time. For the sake of trust and efficiency, we believe in the usefulness of such "longevity" objects, so we reduce the frequency of scanning them in garbage collection.

Little guy needs to check more.

Python divides all the objects into three generations. All new objects are generation 0 objects. When a generation of objects is still alive after garbage collection, it is classified as the next generation of objects. When garbage collection starts, be sure to scan all Gen 0 objects. If Generation 0 passes through a certain number of garbage collections, then start scanning and cleaning of Generation 0 and Generation 1. When the first generation also goes through a certain number of garbage collection, then it will start to scan all objects.

These two times are the two 10s returned by (700,10,10) returned by get_threshold () above. In other words, every 10 times of 0 generation garbage collection will be accompanied by one generation of garbage collection, while for every 10 times of 1 generation garbage collection, there will be 1 generation 2 garbage collection.

You can also use set_threshold () to adjust, such as scanning second-generation objects more frequently.

Import gc gc.set_threshold (700,10,5) isolated reference rings

The existence of reference rings will bring great difficulties to the above garbage collection mechanism. These reference rings may constitute some objects that cannot be used, but the reference count is not zero.

A = [] b = [a] a.append (b) del a del b

Above, we first create two table objects and reference each other to form a reference ring. After removing the aformab reference, it is no longer possible for these two objects to be called from the program, so it is useless. However, due to the existence of the reference ring, the reference count of these two objects is not reduced to 0 and will not be garbage collected.

Isolated reference ring

To recycle such a reference ring, Python copies the reference count for each object, which can be recorded as gc_ref. Suppose that each object I, the count is gc_ref_i. Python iterates through all the objects I. For the object j referenced by each object I, subtract the corresponding gc_ref_j by 1.

The result after traversal

At the end of traversal, objects with non-zero gc_ref, objects referenced by these objects, and objects referenced further down the stream need to be preserved. Other objects are garbage collected.

At this point, the study of "Python memory management example analysis" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.