Shulou(Shulou.com)06/03 Report--

This article mainly introduces "the relationship between Python and GIL mutexes in Python programming". In daily operation, I believe many people have doubts about the relationship between Python and GIL mutexes in Python programming. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "the relationship between Python and GIL mutexes in Python programming". Next, please follow the editor to study!

The Origin of GIL

Python was first released in 1991, when CPU was a single core. In a single core, multithreading was mainly designed to do IO and CPU calculation at the same time. The Python compiler was written by C language, so it was also called CPython. At that time, many programming languages did not have the function of automatic memory management. In order to achieve automatic garbage collection, Python counted references for each object. When the reference count is 0, the object can be recycled, thus freeing up memory, such as:

> import sys > data = {'gzh':' Python No.7'} > var1 = data > sys.getrefcount (data) 3 >

Here the data object has three references, one is itself, one is the variable var1, and the other is the parameter of the getrefcount function. If there is another thread referencing data at this time, then the reference count will be increased by 1. If a thread ends running after using data, then the reference count will be reduced by 1. If multiple threads modify the same variable "reference count", they will encounter race conditions (competition), in order to avoid race conditions. The simplest and most effective way is to add a mutex.

If you lock every object, it may cause another problem, that is, deadlock, and frequent acquisition and release will lead to performance degradation, the most simple and effective way is to add an interpreter lock. Threads acquire interpreter locks when executing any bytecode, which avoids deadlocks and does not consume too much performance. At that time, CPU was a single core, and this GIL design was simple and did not affect performance, so it is still in use today. The main reason for the existence of GIL is that the memory management of Python is not thread safe, which is the main reason for the emergence and existence of GIL.

Try to eliminate GIL

When CPU enters the multi-core era, GIL becomes a real problem when it can do multiple computing tasks at the same time. In 1999, a boss named Greg Stein eliminated GIL based on Python version 1.5, replaced by a finer-grained lock on the variable data structure, and also submitted a patch to remove the dependence on globally variable objects, and then in the standard test shows that after removing GIL, a single thread is nearly twice slower than when it is not removed, and the test machine is still the best Windows machine at that time. That is to say, after removing GIL, you can use 2 CPU to get slightly better performance than the original 1 CPU. This improvement obviously outweighs the gain, and Greg Stein's attempt ends in failure.

Guido van Rossum, the father of Python, also welcomes community volunteers to try to remove GIL, as long as it doesn't degrade single-threaded performance, but he also mentioned that getting rid of GIL is not an easy task.

There are occasional topics on the mailing list of Python developers to remove GIL, but the following requirements must be met:

simple. In the long run, the programme must be implementable and maintainable.

Concurrency. Removing GIL must improve the performance of multithreading.

Speed. Removing GIL does not degrade the performance of a single thread.

Meet the characteristics of CPython. The scheme must support the functions of CPython, such as _ _ del__ and weak references.

Compatibility of API. This scheme should be source compatible with macros used by all existing CPython extensions.

Destroy unreachable objects in time and reclaim memory.

Orderly destruction, such as an unreachable object X that references A, should be destroyed before destroying A (some garbage collection algorithms cannot do this).

Some requirements are not easy to be met, such as 4Magne5 and 7. At present, no one has successfully removed GIL while meeting the above requirements.

confirmed habits are hard to get rid of

Python has been so popular these years, many excellent libraries are written based on CPython, and many of them are C extension libraries from the 1990s. If you want to remove GIL, then many C extensions written based on GIL cannot be used, that is, if you go to GIL,Python, there are many extensions or three-party libraries that cannot be used.

Another obvious example is that Python interpreters not only have CPython, but also use IronPython implemented by Python,.NET written by Java. These interpreters have no GIL at all, but how many people write extensions for them?

The reason why Python is so popular is that it has a great relationship with its rich three-party library to be used out of the box. Long-term accumulation is not easy to solve, and it is difficult to remove GIL.

Why didn't Python3 remove GIL in the first place?

Python3 initially had the opportunity to implement many new features, breaking some existing C extensions in the process, and then needed to update and migrate changes to match Python3, which is why Python3 was not accepted by the community in the first place.

Deleting GIL will make Python3 slower in terms of single-threaded performance than Python2, and many good extensions will no longer be available, and if that's the case, it's conceivable that Python3 has no future, and the end result is that Python3 still has GIL.

However, Python3 has also brought significant improvements to the existing GIL. In version 3.2 of Python, it ensures that when computing-intensive threads and Imax O-intensive threads co-exist, the performance of multithreading is improved due to the long-term inability to obtain GIL.

At this point, the study of "the relationship between Python and GIL mutexes in Python programming" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.