Shulou(Shulou.com)06/01 Report--

This article introduces Python multi-threaded example analysis, the content is very detailed, interested friends can refer to, I hope to help you.

Why do people say Python multithreading is useless? Some people ask such a question. In our common sense, multi-process and multi-thread make full use of hardware resources to improve the running efficiency of programs through concurrency. How can they become chicken ribs in Python?

Some of you may know the answer because of Python's infamous GIL.

So what is GIL? Why GIL? Is multithreading really chicken ribs? Can Gil be removed? With these questions in mind, let's look down together and ask you a little patience.

Multithreading is not chicken ribs, we first do an experiment, the experiment is very simple, that is, the number "100 million" decremented, reduced to 0 on the termination of the program, this task if we use a single thread to execute, how much time will be completed? How much is multithreading? show me the code

#Mission

def decrement(n):

while n > 0:

n -= 1

single-threaded

import time

start = time.time()

decrement(100000000)

cost = time.time() - start

>>> 6.541690826416016

On my 4-core CPU computer, a single thread takes 6.5 seconds. One might ask, where are the threads? In fact, when any program runs, there will be a main thread executing by default. (Threads and processes will not be expanded here, I will open a separate article)

multithreading

import threading

start = time.time()

t1 = threading.Thread(target=decrement, args=[50000000])

t2 = threading.Thread(target=decrement, args=[50000000])

t1.start() #Start thread and execute task

t2.start() #Idem

t1.join() #Main thread blocks until t1 completes execution, main thread continues to execute later

t2.join() #Idem

cost = time.time() - start

>>>6.85541033744812

Create two child threads t1, t2, each thread performs 50 million minus operations, and after both threads are executed, the main thread terminates the program. As a result, the two threads executed cooperatively in 6.8 seconds, which slowed down. Logically speaking, two threads running in parallel on two CPUs at the same time, the time should be halved, now it is not reduced but increased.

What causes multithreading to be slow instead of fast?

The reason is that GIL, in the Cpython interpreter,(Python's mainstream interpreter), there is a global interpretation lock (Global Interpreter Lock), when the interpreter interprets and executes Python code, it must first obtain this lock, which means that only one thread may be executing code at any time. Other threads must obtain this lock first if they want to obtain CPU execution code instructions. If the lock is occupied by other threads, then the thread can only wait. Execution of code instructions is not possible until the thread holding the lock releases the lock.

Therefore, this is why the two threads are slower to execute together, because at the same time, only one thread is running, the other threads can only wait, even if it is a multi-core CPU, there is no way for multiple threads to "parallel" to execute code at the same time, only alternate execution, because multi-threading involves switching on the thread, locking mechanism processing (acquiring locks, releasing locks, etc.), so multi-threaded execution is not fast and slow.

When will GIL be released?

When a thread encounters an I/O task, the GIL is released. Computation-intensive (CPU-bound) threads also release GIL when they execute 100 interpreter ticks (which can be roughly thought of as instructions of the Python virtual machine). You can set the step length by sys.setcheckinterval(), and check the step length by sys.getcheckinterval(). These are mostly the overhead of multithreading compared to single-threading

Why is CPython interpreter designed this way?

Multithreading is to adapt to the rapid development of modern computer hardware to make full use of multi-core processor products, through multithreading so that CPU resources can be efficiently used, Python was born in 1991, when the hardware configuration is far less luxurious than today, now a common server 32 cores 64G memory is not a common thing

But multithreading has a problem, how to solve the synchronization of shared data, consistency problems, because, for multiple threads to access shared data, there may be two threads at the same time modify a data situation, if there is no suitable mechanism to ensure the consistency of data, then the program eventually leads to exceptions, so Python's father has a global thread lock, regardless of whether your data has synchronization problems, anyway, one size fits all, the last global lock, to ensure data security. That's why multithreading is so lame, because it doesn't have fine-grained control over data security, but rather a simple, crude way to solve it.

This solution was put in the 1990s, in fact, there was no problem. After all, the hardware configuration at that time was still very simple, the single-core CPU was still mainstream, and there were not many multi-threaded application scenarios. Most of the time, it was still running in a single-threaded manner. Single-threaded threads did not involve context switching of threads, but the efficiency was higher than multi-threaded threads (in a multi-core environment, this rule did not apply). Therefore, the use of GIL to ensure data consistency and security is not necessarily undesirable, at least at the time is a very low cost implementation.

Is it possible to remove GIL?

Some people do, but the results are disappointing, and in 1999 Greg Stein and Mark Hammond created a Python branch that removed GIL and replaced it with more fine-grained locks on all variable data structures. However, after benchmarking, Python without the GIL executes nearly twice as slowly in single-threaded conditions.

Python's father says that, based on the above considerations, there's not much value in getting rid of GIL without spending too much effort.

The example analysis of Python multithreading is shared here. I hope the above content can be of some help to everyone and learn more. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.