Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the multithreading and multiprocess of Python". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what is the multithreading and multiprocess of Python"?

Premise

I wrote it with reference to Github Python's 100th day article, and combined with my own small exercises, I summarized

Recently, in the noodle factory, I found that many big factories will ask the multi-thread and multi-process of Python, so I think it is necessary to sum up and learn.

What is a process?

A program executed in an operating system, similar to Wechat or QQ, in which each program is a process

Concept

It is the minimum resource allocation unit of CPU

The operating system allocates memory space to processes, and each process has its own address space, data stack, and other auxiliary data for tracking process execution.

The operating system manages the execution of all processes and allocates resources to them reasonably

Fork 、 spawn

Processes can create new processes to perform other tasks through fork and spawn.

But the new process has its own independent memory space and data stack.

Therefore, different processes need to use communication mechanism (IPC) to achieve data sharing.

Common communication mechanism

Pipeline

Signal

Socket

Shared memory area

What is a thread?

There can be multiple concurrent execution clues in a process

It is the execution unit of CPU minimum scheduling.

Characteristics

Threads under the same process share the same context

Compared with processes, information sharing and communication between threads is easier.

Considerations for single-core CPU system

True concurrency is impossible.

Because at some point, CPU can only run a single thread

Multiple threads share the execution time of CPU

Benefits of multithreading

Improve program performance and user experience

Almost all the software used in daily use today uses multithreading.

The disadvantages of multithreading

From the point of view of other processes, multithreaded programs are not friendly to other programs because they take up more CPU execution time, resulting in other programs not getting enough CPU execution time.

It is highly demanding for developers to write and debug multithreaded programs.

The way of realizing concurrent programming with Python

Multiple processes

Multithreading

Multi-process + multi-thread

Fork function under Multi-process Linux in Python

The fork () system call is provided on the Linux operating system to create processes.

It is the parent process that calls the fork () function

The child process is created

The child process is a copy of the parent process

But the child process has its own PID

The fork () function is very special. It returns twice. Calling fork () in the parent process returns the PID of the child process, and calling fork () in the child process gets 0.

Fork function provided by Python

The os module provides fork ()

There is no call to fork () under Window

To achieve cross-platform multi-processes, you can use the Process class of the multiprocessing module to create child processes

More advanced encapsulation is also provided, such as process pool pool for batch startup processes, queue Queue for sharing love between processes, and pipeline Pipe.

The difference between using multi-process and not using multi-process (writing code) does not use multi-process from random import randintfrom time import time, sleepdef download_task (filename): print ('start downloading% s..% filename) time_to_download = randint (5,10) sleep (time_to_download) print ('% s download complete! Spent% d seconds'% (filename Time_to_download)) def main (): start = time () download_task ('Python from entry to hospitalization. Pdf') download_task ('Peking Hot.avi') end = time () print (' took a total of% .2f seconds.% (end-start)) if _ name__ = ='_ main__': main () the result starts downloading Python from entry to hospitalization. Pdf... Python from entry to hospitalization. pdf download completed! It took 10 seconds to start downloading Peking Hot.avi...Peking Hot.avi download complete! It took 9 seconds and a total of 19.02 seconds.

You can see that you need to wait for the first file to download before you can download the second file, which is very inefficient.

Using multiprocess from random import randintfrom time import time, sleepfrom multiprocessing import Processdef download_task (filename): print ('start downloading% s.'% filename) time_to_download = randint (5,10) sleep (time_to_download) print ('% s download complete! Spent% d seconds'% (filename, time_to_download) def main2 (): start = time () p1 = Process (target=download_task,args= ("Python from entry to hospitalization .pdf",) p1.start () p2 = Process (target=download_task,args= ("Peking Hot.avi") ) p2.start () p1.join () p2.join () end = time () print ('took a total of% .2f seconds.% (end-start)) if _ _ name__ = =' _ main__': main2 () the execution result starts downloading Python from entry to hospitalization. Pdf... Start downloading Peking Hot.avi...Python from entry to hospitalization. pdf download complete! It took 6 seconds to complete the Peking Hot.avi download! It took 10 seconds and a total of 10.17 seconds.

When two tasks are executed at the same time, the total time spent is no longer the sum of the time of the two tasks.

Knowledge point

Process: creating process objects through the Process class

Target: pass in a function name through the target parameter to represent the code to be executed after the process starts

Args: is a tuple that represents the list of arguments passed to the function

The start () method of start:Process to start the process

The join () method of join:Process indicates that it waits for the execution of the process to finish before it will proceed.

Multithread preface in Python

Threading module is recommended for multithreaded programming, which provides better object-oriented encapsulation.

The implementation of multithreading is #! / usr/bin/env python#-*-coding: utf-8-*-"" _ _ title__ = _ _ Time__ = 2021-3-19 1818 title__ authorizations _ = small pineapple test notes _ _ Blog__ = https://www.cnblogs.com/poloyy/"""from random import randintfrom threading import Threadfrom time import time Sleepdef download_task (filename): print ('start downloading% s..'% filename) time_to_download = randint (5,10) sleep (time_to_download) print ('% s download complete! Spent% d seconds'% (filename, time_to_download) def main3 (): start = time () p1 = Thread (target=download_task,args= ("Python from entry to hospitalization .pdf",) p1.start () p2 = Process (target=download_task,args= ("Peking Hot.avi") ) p2.start () p1.join () p2.join () end = time () print ('took a total of% .2f seconds.% (end-start)) if _ _ name__ = =' _ main__': main3 () the execution result starts downloading Python from entry to hospitalization. Pdf... Start downloading Peking Hot.avi...Peking Hot.avi download complete! It took 6 seconds to complete the Python from entry to hospitalization. It took 8 seconds and a total of 8.01 seconds.

The same execution efficiency is much higher.

Custom thread class #! / usr/bin/env python#-*-coding: utf-8-*-"" _ _ title__ = _ _ Time__ = 2021-3-19 18 purse 17mm authorizations _ = small pineapple test notes _ _ Blog__ = https://www.cnblogs.com/poloyy/"""from random import randintfrom threading import Threadfrom time import time, sleepclass downLoadTask (Thread): def _ _ init__ (self) Filename): super (). _ init__ () self.filename = filenamedef run (self)-> None:print ('start downloading% s..% self.filename) time_to_download = randint (5,10) sleep (time_to_download) print ('% s download complete! Spent% d seconds'% (self.filename Time_to_download)) def main3 (): start = time () p1 = downLoadTask ("Python from entry to hospitalization .pdf") p2 = downLoadTask ("Peking Hot.avi") p1.start () p2.start () p1.join () p2.join () end = time () print ('took% .2f seconds.% (end-start)) if _ name__ =' _ main_ _': main3 () execution result starts downloading Python from entry to hospitalization. Pdf... Start downloading Peking Hot.avi...Peking Hot.avi download complete! It took 6 seconds to complete the Python from entry to hospitalization. It took 9 seconds and a total of 9.00 seconds.

It's the same efficient operation.

Key knowledge: the difference between start and run method comparison point startrun function starts the thread, obtains the CPU time slice running thread specified code block thread state runnable state running state call times a thread can only be called once, the running thread can be called repeatedly to create a child thread, the thread name is named by itself, and a common function is called in the main thread. Note that you want to use multiple threads. Must call the co-program in start () Python. What is the co-program?

Programming Model of single Thread + Asynchronous Igamot O in Python

Advantages of Synergetics

Extremely high execution efficiency

Subroutine switching is not thread switching, but is controlled by the program itself, and there is no thread switching overhead.

There is no need for the mechanism of multithreading, there is only one thread, so there is no conflict between writing variables at the same time, and there is no need to lock the shared resources in the cooperative program, only to judge the state, so the execution efficiency is much higher than that of multithreading.

Highlight

In order to make full use of the multi-core characteristics of CPU, the method of multi-process + cooperative program should be used.

Thank you for your reading, these are the contents of "what is the multithreading and multiprocess of Python". After the study of this article, I believe you have a deeper understanding of what the multithreading and multiprocess of Python is, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.