Shulou(Shulou.com)06/02 Report--

This article mainly explains "multithreading and multiprocessing analysis in Python". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let Xiaobian take you to learn "multithreading and multiprocessing analysis in Python"!

multithreading

Simply put, threads allow you to run programs in parallel. Tasks that spend a lot of time waiting for external events are usually good candidates for threading. They are also known as I/O Bound tasks such as reading and writing from files, network operations or downloading online using APIs. Let's look at an example that demonstrates the benefits of using threads.

1. no Thread

In this example, we want to see how long it takes to download 15 images from the Unsplash API by running the program sequentially:

import requests import time img_urls = [ 'https://images.unsplash.com/photo-1516117172878-fd2c41f4a759', 'https://images.unsplash.com/photo-1532009324734-20a7a5813719', 'https://images.unsplash.com/photo-1524429656589-6633a470097c', 'https://images.unsplash.com/photo-1530224264768-7ff8c1789d79', 'https://images.unsplash.com/photo-1564135624576-c5c88640f235', 'https://images.unsplash.com/photo-1541698444083-023c97d3f4b6', 'https://images.unsplash.com/photo-1522364723953-452d3431c267', 'https://images.unsplash.com/photo-1513938709626-033611b8cc03', 'https://images.unsplash.com/photo-1507143550189-fed454f93097', 'https://images.unsplash.com/photo-1493976040374-85c8e12f0c0e', 'https://images.unsplash.com/photo-1504198453319-5ce911bafcde', 'https://images.unsplash.com/photo-1530122037265-a5f1f91d3b99', 'https://images.unsplash.com/photo-1516972810927-80185027ca84', 'https://images.unsplash.com/photo-1550439062-609e1531270e', 'https://images.unsplash.com/photo-1549692520-acc6669e2f0c' ] start = time.perf_counter() #start timer for img_url in img_urls: img_name = img_url.split('/')[3] #get image name from url img_bytes = requests.get(img_url).content with open(img_name, 'wb') as img_file: img_file.write(img_bytes) #save image to disk finish = time.perf_counter() #end timer print(f"Finished in {round(finish-start,2)} seconds") #results Finished in 23.101926751 seconds

It took 23 seconds.

2. multithreading

Let's see how the threading module in Pyhton significantly improves our program execution:

import time from concurrent.futures import ThreadPoolExecutor def download_images(url): img_name = img_url.split('/')[3] img_bytes = requests.get(img_url).content with open(img_name, 'wb') as img_file: img_file.write(img_bytes) print(f"{img_name} was downloaded") start = time.perf_counter() #start timer with ThreadPoolExecutor() as executor: results = executor.map(download_images,img_urls) #this is Similar to map(func, *iterables) finish = time.perf_counter() #end timer print(f"Finished in {round(finish-start,2)} seconds") #results Finished in 5.544147536 seconds

We can see that using threaded code can significantly improve speed compared to not using threaded code. From 23 seconds to five seconds.

For this example, note that there is overhead in creating threads, so it makes sense to use threads for multiple API calls, not just a single call.

Furthermore, for intensive computations such as data processing, image processing multiprocessing performs better than threading.

At this point, I believe that everyone has a deeper understanding of "multithreading and multiprocessing analysis in Python", so let's actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.