Shulou(Shulou.com)05/31 Report--

Today, the editor will share with you how to use Python to create a batch download video and visualize the download progress of the downloader related knowledge points, detailed content, clear logic, I believe that most people still know too much about this knowledge, so share this article for your reference, I hope you can learn something after reading this article, let's take a look at it.

[1. Project background]

The most favorite thing we do at home is to watch TV dramas, but sometimes, for Internet reasons, you may not have access to the Internet, and those good TV dramas and movies are naturally impossible to watch at this time. What we are going to talk about in this issue is how to download these videos.

[II. Project objectives]

Download the videos you are interested in in bulk through the Python program, just as the editor has seen some good videos recently, because he wants to go in the direction of Android, but it is too complicated to re-learn, is there anything simple, it seems that there was some E4a but gave up learning the easy language, so I found a minority programming language-Yu language in the vast network. Boy, stop talking, download it and disk it.

[III. Project implementation]

Using sublime text 3 to write a program, first look at the effect:

[IV. Implementation steps]

1. Analyze the structure of web pages

As usual, review the element positioning, as shown in the following figure:

Found that the videos are all in the a tag, because there are more videos on this page, so we continue to analyze the page and find a magical thing. Haha, it turns out that all the videos are in the a tag of class for videoDown. With this important information, everything will be easy.

# parsing page def parser (): ab= [] rep=requests.get ('http://v.u00.cn:93/iappce.htm#sp',timeout=5,headers=headers) rep.encoding='utf-8' soup=BeautifulSoup (rep.text,'html.parser') res=soup.find_all (' a') Class_='videoDown') # find all the a tags for y in res: ab.append whose class is videoDown ('http://v.u00.cn:93'+y.attrs['href']) # adds the obtained video URL address to the list return ab # returns the list of all video addresses

In this way, you can easily get all the video addresses on the page. How is it super easy?

two。 Download a file

Because we are talking about bulk downloads, we need to understand individual downloads before that, of course, individual downloads are time-consuming and the system resource utilization is too low.

Let's see how this download function is implemented:

# download function def down (yy.split x): print ('- download', str (x), 'lesson -') ss=str (y.split ('.) [3:4])\ # intercept the file name sa=ss.replace ('[','). Replace ('],')\ # replace the special symbol ree=requests.get (y) with open ('% d% s.mp4percent) in the file name 'wb') as f: f.write (ree.content)\ # Save the file

Nothing more than some commonly used string separation and file operations, but because this is relatively simple, it is not feasible to download multiple files, so generally as long as it is not a mass download, this method is enough.

And then give him a function to simplify his startup path.

Def main (): for y in range (len (parser (): down (parser () [y], y)\ # download main ()

Finally, the main function main is called to download a single file easily.

3. Get the file size and buffer the downloaded file

When downloading a video, if we take all the resources out at once and put them into CPU to read, then we will crash very quickly, so we need to set up a buffer, wait for his buffer to be full, and then take it out to read. It sounds abstract. Let's take a look at it.

The values shown in the figure are the video size value and the request range value.

1. Get the video size

Def download (url, file_name):\ # download video urllib3.disable_warnings () rep=requests.get (url,headers=headers) head=rep.headers rep=requests.get (url,headers=headers) head=rep.headers\ # get the request header dictionary length=head.get ('Content-Length')\ # get the size of the video in bytes if you want him to use MB as the unit Divide by two 1024 file_size = int (length)\ # convert the video size with int if os.path.exists (file_name):\ # determine whether the file name exists first_byte = os.path.getsize (file_name)\ # get the file name size else: first_byte = 0 if first_byte\ > = file_size: return file_size\ # return the file size Header = {"Range": "bytes=%s-%s"% (first_byte File_size), 'user-agent':' Mozilla/5.0 (Windows NT 6.1) WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'}\ # set the request header to indicate the request scope

two。 Configure the progress bar

Pbar = tqdm (\ # configure progress bar module, set file size, file bytes, file progress total=file_size, initial=first_byte, unit='B', unit_scale=True, desc=url.split ('/') [- 1]) # you can use Baidu tqdm module for specific usage of tqdm.

3. Add buffer

With closing (requests.get (url, headers=header, stream=True)) as req: # close connection with open (file_name 'wb') as f:\ # Open file for chunk in req.iter_content (chunk_size=1024\ * 2):\ # set buffer if chunk: pbar.set_description ("[downloading video% s]"% str (f.name)) f.write (chunk)\ # write to the file Pbar.update (1024)\ # Update the current progress bar pbar.close ()\ # close the progress bar return file_size\ # return the file size

4. Build the download video and display the progress bar function

Def fd ():\ # download and display the progress bar global x xpen1 for y in parser (): print ('- downloading', xmeme 'lesson -') ss=str (y.split (.) [3:4]) sa=ss.replace ('[','). Replace (']',)\ # filename download (y "{}. {} .mp4" .format (str (x), sa))\ # download print ('- finish downloading', XMagi 'lesson -') xresume 1

5. Startup program

Fd () above is all the contents of the article "how to use Python to create a downloader for downloading videos in bulk and visualizing the download progress". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.