Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge of "what is concurrent programming on the basis of python". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

I. definition and function of collaborative process

Coroutine, also known as microthreading, fiber. (a co-program is a lightweight thread in user mode.)

Function: when executing the A function, you can interrupt at any time to execute the B function, and then interrupt to continue to execute the A function (which can be switched automatically). A single procedure is not a function call (no calling statement), and the process is very much like multithread. however, there is only one thread executing the co-program.

1. Advantages of using cooperative programs

Because it has context and stack, it does not need the overhead of thread context switching, and belongs to program-level switching, which is completely imperceptible to the operating system, so it is more lightweight.

No locking and synchronization overhead without atomic operations

Easy to switch control flow and simplify programming model

Concurrency can be achieved within a single thread, making maximum use of cpu, with high scalability and low cost. (note: it is not a problem to support tens of thousands of collaborations per CPU. So it is very suitable for high concurrency processing)

2. Disadvantages of using cooperative programs.

Unable to make use of multi-core resources: the essence of the cooperative program is a single thread, it can not use multiple cores of a single CPU at the same time, the cooperative program needs to cooperate with the process to run on the multi-CPU. Of course, most of the applications we write every day don't have this requirement, except for cpu-intensive applications.

Performing a Blocking operation, such as IO, will block the entire program

# basic use of collaborative programs Switch between yield and next to switch back and forth between def func1 (): for i in range (11): print (f "first class printing {I} data") yielddef func2 (): G = func1 () next (g) for i in range (10): print (f "second class printing {I} data") next (g) if _ _ name__ = "_ _ main__": func2 () II. The use of Greenlet

There are multiple tasks in a single thread. Greenlet is used to switch tasks and combine greenlet and switch.

From greenlet import greenlet# pip install greenletdef gf (name): print (f'{name}: I want Masters!) G2.switch ('zf') print (f' {name}: I want a big meal!') G2.switch () def bf (name): print (f'{name}: go together!') G1.switch () print (f'{name}: let's eat!') If _ _ name__ = = "_ main__": G1 = greenlet (gf) G2 = greenlet (bf) # switching task g1.switch ('dc') # only need to upload the third for the first time, the use of Gevent

Gevent is a third-party library that can easily implement concurrent synchronous or asynchronous programming through gevent. The main mode used in gevent is Greenlet, which is a lightweight protocol connected to Python in the form of C extension modules.

Greenlet all run inside the main operating system processes, but they are scheduled in a collaborative manner.

From gevent import monkey; # in order to identify the iomonkey.patch_all () # of the time module, it must be placed in front of the patched, such as import gevent# pip install geventfrom time import time,sleepdef gf (name): print (f'{name}: I want to play Masters!) before the time,socket module. # gevent.sleep (2) sleep (2) print (f'{name}: I want a big meal!') Def bf (name): print (f'{name}: play together!') # gevent.sleep (2) sleep (2) print (f'{name}: go and eat!) If _ _ name__ = = "_ main__": start = time () # create a cooperative program object G1 = gevent.spawn (gf,' Diao Chan') G2 = gevent.spawn (bf,' Lv Bu') # Open task g1.join () g2.join () end = time () print (end-start)

Note: the above example gevent.sleep (2) simulates io blocking that gevent can recognize; while time.sleep (2) or other blocking cannot be directly recognized by gevent, you need to use the following line of code to patch it.

IV. Async io Asynchronous IO

Asyncio is the co-program module after python3.4, and it is an important package for python to implement concurrency. This package uses event loop driver to achieve concurrency.

Event loop is an effective way to deal with multiple concurrency. In Wikipedia, it is described as "a programming architecture that waits for programs to assign events or messages". We can define event loops to simplify the use of polling methods to monitor events. The popular saying is "execute B when An occurs".

@ asyncio.coroutine co-program decorator decoration

Asyncio.sleep () can avoid event loop blocking

Get_event_loop () gets the event loop

Loop.run_until_complete () listens for event loops

Gather () encapsulation task

Await equals yield from is waiting for the task result.

Import asyncio@asyncio.coroutine # python3.5 said on the official website that 3.10 will be removed def func1 (): for i in range (5): print ('while eating!') Yield from asyncio.sleep (0) async def func2 (): # python3.5 above for i in range (5): print ('while playing games!') Await asyncio.sleep (0) if _ _ name__ = = "_ main__": G1 = func1 () G2 = func2 () # get event loop loop = asyncio.get_event_loop () # listen event loop loop.run_until_complete (asyncio.gather (G1 dint G2)) # close event loop loop.close () 1, use import asyncioimport functoolsasync def compute (x) of task in asyncio Y): print (f'compute: {x} + {y}....') Await asyncio.sleep (1) return x+yasync def print_sum (XMagi y): # create task task = asyncio.create_task (compute (XMago y)) # python3.7 above # task bind callback function task.add_done_callback (functools.partial (end,x) Y)) # python3.7 above # release the use of cpu await asyncio.sleep (0) print ('- print_num continue to execute -) for i in range (1000000): if I% 5000 = = 0: print (I) await asyncio.sleep (0.1) def end (n M main__ t): print (f'{n} + {m} = {t.result ()}') if _ _ name__ = = "_ _ main__": loop = asyncio.get_event_loop () loop.run_until_complete (print_sum (1)) loop.close () "what is the basis of python concurrent programming" ends here Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.