Shulou(Shulou.com)06/02 Report--

This article mainly introduces "what is the concept of Python process and thread". In daily operation, I believe that many people have doubts about the concept of Python process and thread. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "what is the concept of Python process and thread". Next, please follow the editor to study!

Concurrent programming is to enable programs to perform multiple tasks at the same time, but how to achieve concurrent programming? this involves the two concepts of process and thread.

For the operating system, a task (or program) is a Process. For example, opening a browser opens a browser process, opening Wechat starts a Wechat process, and opening two notepads starts two notepad processes.

The characteristics of the process are:

The operating system allocates storage space per process, and each process has its own address space, data stack, and other auxiliary data used to track the execution of the process.

Processes can create new processes through fork or spawn to perform other tasks

All processes have their own independent memory space, so processes need to share data through inter-process communication mechanism (IPC,Inter-Process Communication), including pipes, signals, sockets, shared memory areas, etc.

A process can also do many things at the same time, such as typing, pinyin checking, printing and other things at the same time in Word, that is, a task is divided into multiple subtasks at the same time, and these subtasks in the process are called Thread.

Because each process needs to accomplish at least one thing, that is, a process has at least one thread. When it comes to concurrent programming, that is, multitasking, there are three solutions:

Multiple processes, each process has only one thread, but multiple processes perform multiple tasks together

Multithreading, starting only one process, but opening multiple threads within a process

Multi-process + multi-thread, that is, start multiple processes, and each process starts multiple threads, but this method is very complex and rarely used in practice.

Note: true parallel execution of multitasking can only be implemented on multi-core CPU. In a single-core CPU system, true concurrency is impossible, because at some point there is only one thread that can get CPU, and multiple threads share the execution time of CPU.

Python supports both multiprocess and multithreading, which are described below.

Multiple processes

In the Unix/Linux system, a fork () system call is provided, which is a special function. The ordinary function call is called once and returns once, but the fork function is called once and returns twice, because the function is called by the parent process, and then a child process is copied. Finally, it is returned in both the parent process and the child process, so it will be returned twice.

The child process always returns 0, and the parent process returns the ID of the child process, because the parent process can copy multiple child processes, so you need to record the ID of each child process, and the child process can get the ID of the parent process by calling getpid ().

The os module in Python encapsulates common system calls, including fork. The code example is as follows:

Import osprint ('Process (% s) start...'% os.getpid ()) # Only works on Unix/Linux/Mac:pid = os.fork () if pid = 0: print (' I am child process (% s) and my parent is% s.% (os.getpid (), os.getppid ()) else: print ('I (% s) just created a child process (% s).% (os.getpid (), pid)

Running result:

Process start...I 876 just created a child process. I am child process 877 and my parent is 876.

Since fork does not exist in the windows system, the above functions cannot be called, but Python is cross-platform, so there are other modules that can achieve multi-process functions, such as the multiprocessing module.

Multiprocess

The Process class is provided in the multiprocessing module to represent a process object, and then an example of downloading a file is used to illustrate the difference between using multiple processes and not using multiple processes.

The first is the example of not using multiple processes:

Def download_task (filename):''simulated download file' 'print (' start downloading% s.% filename) time_to_download = randint (5,10) sleep (time_to_download) print ('% s download complete! % d seconds'% (filename, time_to_download) def download_without_multiprocess ():''do not use multiple processes''start = time () download_task (' Python.pdf') download_task ('nazha.mkv') end = time () print (' total .2f seconds.% (end-start)) if _ name__ = ='_ main__': download_without_multiprocess ()

The running results are as follows. Here, the randint function is used to randomly output the time spent on the current download file. From the result, the running time of the program is equal to the sum of the task time of the two download files.

Start downloading Python.pdf...Python.pdf download complete! It took 9 seconds to start downloading nazha.mkv...nazha.mkv download complete! It took 9 seconds and a total of 18.00 seconds.

If you are using multiple processes, the example is as follows:

Def download_task (filename):''simulated download file' 'print (' start downloading% s.% filename) time_to_download = randint (5,10) sleep (time_to_download) print ('% s download complete! Spent% d seconds'% (filename, time_to_download) def download_multiprocess ():''adopt multiple processes' 'start = time () p1 = Process (target=download_task, args= (' Python.pdf',)) p1.start () p2 = Process (target=download_task, args= ('nazha.mkv') ) p2.start () p1.join () p2.join () end = time () print ('spent% .2f seconds.% (end-start)) if _ _ name__ = =' _ _ main__': download_multiprocess ()

In this multi-process example, we create a process object through the Process class, and pass in a function to represent the task that the process needs to execute through the target parameter. Args is a tuple that represents the parameter passed to the function, and then uses start to start the process, while the join method indicates waiting for the process to finish execution.

The running results are as follows, the time consuming is not the sum of the execution time of the two tasks, and the speed is greatly improved.

Start downloading Python.pdf... Start downloading nazha.mkv...Python.pdf download complete! It took 5 seconds to complete the nazha.mkv download! It took 9 seconds and a total of 9.36 seconds.

Pool

In the above example, two processes are started, but if you need to open a large number of child processes, the above code is not appropriate. You should use the process pool to create child processes in batch, or use the example of downloading files. But the code that executes the next part is as follows:

Import osfrom multiprocessing import Process, Poolfrom random import randintfrom time import time, sleepdef download_multiprocess_pool ():''adopt multiple processes And use pool to manage the process pool''start = time () filenames = [' Python.pdf', 'nazha.mkv',' something.mp4', 'lena.png',' lol.avi'] # process pool p = Pool (5) for i in range (5): p.apply_async (download_task, args= (filenames [I]) ) print ('Waiting for all subprocesses done...') # close the process pool p.close () # wait for all processes to complete the task p.join () end = time () print (' took% .2f seconds.% (end-start)) if _ _ name__ = ='_ _ main__': download_multiprocess_pool ()

In the code, the Pool object first creates five processes, and then the apply_async method starts the process to execute the task in parallel. Before calling the join () method, you must first call close (), and close () mainly closes the process pool, so you cannot add new process objects after executing this method. Then join () just waits for all the processes to finish the task.

The running result is as follows:

Waiting for all subprocesses done... Start downloading Python.pdf... Start downloading nazha.mkv... Start downloading something.mp4... Start downloading lena.png... Start downloading lol.avi...nazha.mkv download complete! It took 5 seconds to complete the lena.png download! It took 6 seconds to complete the something.mp4 download! It took 7 seconds to complete the Python.pdf download! It took 8 seconds to complete the lol.avi download! It took 9 seconds and a total of 9.80 seconds.

Child process

In most cases, the child process is an external process, not itself. After creating the child process, we also need to control the input and output of the child process.

The subprocess module allows us to start child processes and manage the input and output of child processes.

The following is a demonstration of how to demonstrate the command nslookup www.python.org with Python, with the following code:

Import subprocessprint ('$nslookup www.python.org') r = subprocess.call (['nslookup',' www.python.org']) print ('Exit code:', r)

Running result:

$nslookup www.python.orgServer: 192.168.19.4Address: 192.168.19.4#53Non-authoritative answer:www.python.org canonical name = python.map.fastly.net.Name: python.map.fastly.netAddress: 199.27.79.223Exit code: 0

If the child process needs input, it can be entered through communicate (), as shown in the following code:

Import subprocessprint ('$nslookup') p = subprocess.Popen (['nslookup'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) output, err= p.communicate (b'set q=mx\ npython.org\ nexit\ n') print (output.decode ('utf-8')) print (' Exit code:', p.returncode)

This code is when you execute the command nslookup, type:

Set q=mxpython.orgexit

Running result:

$nslookupServer: 192.168.19.4Address: 192.168.19.4#53Non-authoritative answer:python.org mail exchanger = 50 mail.python.org.Authoritative answers can be found from:mail.python.org internet address = 82.94.164.166mail.python.org has AAAA address 2001:888:2000:d::a6Exit code: 0

Interprocess communication

Communication is needed between processes, and Queue, Pipes and other ways are also provided in the multiprocess module to exchange data.

Here, take Queue as an example, create two child processes in the parent process, one writing data to Queue and the other reading data from Queue. The code is as follows:

Import osfrom multiprocessing import Process, Queueimport randomfrom time import time, sleep# write the code executed by the data process: def write (Q): print ('Process to write:% s'% os.getpid ()) for value in ['A','B' 'C']: print (' Put% s to queue...'% value) q.put (value) sleep (random.random ()) # Code executed by the read data process: def read (Q): print (% Process to read:% s'% os.getpid ()) while True: value = q.get (True) print ('Get% s from queue.'% value) def ipc_queue ():' 'use Queue to achieve interprocess communication Letter: return:''# parent process creates Queue And passed to each child process: Q = Queue () pw = Process (target=write, args= (Q,)) pr = Process (target=read, args= (Q,)) # Promoter process pw, write: pw.start () # Promoter process pr, read: pr.start () # wait for pw to end: pw.join () # pr process is an endless loop and cannot wait for it to end Can only be forcibly terminated: pr.terminate () if _ _ name__ = ='_ main__': ipc_queue ()

The running result is as follows:

Process to write: 24992Put A to queue...Process to read: 22836Get A from queue.Put B to queue...Get B from queue.Put C to queue...Get C from queue. At this point, the study of "what is the concept of Python processes and threads" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.