How to write the source code of bilibili's mini video crawled by Python crawler
Today, I will talk to you about how to write the source code of the mini video of Python crawler crawling bilibili. Many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.
If you want to climb multiple pages, just fill in the number of times of the loop in the bottom loop.
Project source code
From fake_useragent import UserAgentimport requestsimport time
Ua=UserAgent ()
Def downloader (url, path): start = time.time () # start time size = 0 headers= {'User-Agent':ua.random} response = requests.get (url, headers=headers Stream=True) # stream attribute must be chunk_size = 1024 # data size per download content_size = int (response.headers ['content-length']) # Total size if response.status_code = = 200: print (' [file size]:% 0.2f MB'% (content_size / chunk_size / 1024)) # conversion unit with open (path) 'wb') as file: for data in response.iter_content (chunk_size=chunk_size): file.write (data) size + = len (data) # downloaded file size print ('\ r'+'[download progress]:% s% .2f%%'% ('>'* int (size * 50 / content_size)) Float (size / content_size * 100), end= "") end= time.time () # end time print (\ n' + 'video download completed! Time% .2f second'% (end-start))
Def The_URL (page): URL=' http://api.vc.bilibili.com/board/v1/ranking/top?page_size=10&next_offset={}&tag=%E4%BB%8A%E6%97%A5%E7%83%AD%E9%97%A8&platform=pc'.format(page) headers= {'User-Agent':ua.random} sponse=requests.get (URL Headers=headers) .json () item=sponse.get ('data') .get (' items') for i in item: ite=i.get ('item') # video title Video_name=ite.get (' description')
# release date Release_time=ite.get ('upload_time_text')
# Video download address Video_download_link=ite.get ('video_playurl')
# Video author The_name=i.get ('user'). Get (' name')
Try: print ('currently downloaded is:% s'%Video_name) downloader (Video_download_link,path='%s.mp4'%Video_name) except Exception as e: print (e.args)
For i in range (0100): i=i*10+1 The_URL (I) after watching the above, do you have any further understanding of how to write the source code of the mini video of Python crawler crawling bilibili? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.
