Shulou(Shulou.com)06/01 Report--

This article mainly introduces how python climbs the popular video of bilibili. The article is very detailed and has certain reference value. Interested friends must finish watching it!

The text and pictures of this article come from the Internet, are for study and communication only, and do not have any commercial use. The copyright belongs to the original author. If you have any questions, please contact us in time for handling.

The code is as follows

Import requestsfrom lxml import etreeimport xlwtimport os# crawled bilibili's popular video information def spider (): video_list = [] url = "https://www.bilibili.com/ranking?spm_id_from=333.851.b_7072696d61727950616765546162.3" html = requests.get (url, headers= {" User-Agent ":" Mozilla/5.0 (Windows NT 10.0) WOW64) AppleWebKit/537.36 (KHTML Like Gecko) Chrome/75.0.3770.100 Safari/537.36 "}) .text html = etree.HTML (html) infolist = html.xpath (" / / li [@ class='rank-item'] ") for item in infolist: rank =" .join (item.xpath (". / div [@ class='num'] / text ()") video_link = "" .join (item.xpath (". / / div [@ class='info'] / aside @) Href ") title =" .join (item.xpath (". / / div [@ class='info'] / a/text ()") payinfo = ".join (item.xpath (". / / div [@ class='detail'] / span/text () "). Split (" ten thousand ") play = payinfo [0] +" ten thousand "comment = payinfo [1] if comment.isdigit () = False: comment + =" ten thousand " Upname = ".join (item.xpath (". / / div [@ class='detail'] / a/span/text () ") uplink =" http://" + ".join (item.xpath (". / / div [@ class='detail'] / a/@href ") hot =".join (item.xpath (". / / div [@ class='pts'] / div/text () ") video_list.append ({'rank': rank) 'videolink': video_link, 'title': title,' play': play, 'comment': comment,' upname': upname, 'uplink': uplink 'hot': hot}) return video_listdef write_Excel (): # add crawled information to Excel video_list = spider () workbook = xlwt.Workbook () # define table sheet = workbook.add_sheet ("bilibili popular video") # add name xstyle = xlwt.XFStyle () # instantiated table style object xstyle.alignment.horz = 0x02 # font centered xstyle. Alignment.vert = 0x01 head = ['video name' 'up Master', 'ranking', 'Hot', 'number of views', 'number of comments'] for h in range (len (head)): sheet.write (0, h, head [h] Xstyle) I = 1 for item in video_list: # add the hyperlink to the video if'"'in item [" title "]: item [" title "] = item [" title "] .split ('") [1] title_data = 'HYPERLINK ("' + item [" videolink "] +'" "'+ item [" title "] +'")'# set the hyperlink sheet.col (0). Width = int (256xxxxxxxxxxxxxxxxxxxxxxxxxxxxxmxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "'+ item [" upname "] +') 'sheet.col (1). Width = int (256 * len (name_data) * 3 sheet.write 5) sheet.write (I, 1, xlwt.Formula (name_data), xstyle) sheet.write (I, 2, item [" rank "], xstyle) sheet.write (I, 3, item [" hot "], xstyle) sheet.write (I, 4, item [" play "], xstyle) sheet.write (I, 5) Item ["comment"], xstyle) I + = 1 # if the file exists Delete it file = "bilibili popular video information .xls" if os.path.exists (file): os.remove (file) workbook.save (file) if _ _ name__ ='_ _ main__': write_Excel ()

The results show:

The above is all the content of this article "how python crawls bilibili's popular video". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.