How to crawl HD desktop wallpaper by Python
Python how to crawl high-definition desktop wallpaper, in view of this problem, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and easy way.
Just run it directly, and the effect picture:
The code is as follows
Import tool
Import requests,osfrom pyquery import PyQuery as pqimport time
Request and parse web page data, save data
Headers = {'User-Agent':' Mozilla/5.0 (Macintosh Intel Mac OS X 10'11'4) AppleWebKit/537.36''(KHTML Like Gecko) Chrome/52.0.2743.116 Safari/537.36'} # here I use the proxy you can remove this proxy IP I am preparing for the later large-scale crawling proxies = {'https':' 218.75.69.50 proxies 39590'} # request the web page to get the source code def start_request (url): r = requests.get (url, headers=headers) Proxies=proxies) # the page of this website uses GBK encoding here to transcode r.encoding = 'GBK' html = r.text return html # parse the page to get the picture def parse (text I): doc = pq (text) # Lock the img tag images = doc ('div.list ul li img'). Items () x = 1 for image in images: # get the link to each picture img_url = image.attr (' src') # get the binary content of each picture img = requests.get (img_url, headers=headers) Proxies=proxies) .content # define Lu Jin # create directory dir_path = os.path.dirname (os.path.abspath (_ _ file__)) + "\\ image\\" imagspath = dir_path+'/imgs'+str (I) # create directory if not os.path.exists (imagspath): os.makedirs (imagspath) Path = imagspath+ "\\" + str (x) + ".jpg" # write the picture to the specified directory and write to the file with "wb" with open (path 'wb') as f: f.write (img) time.sleep (1) print ("downloading {} picture on page {}" .format (iMagazine x)) x + = 1 print ("write complete") def main (): url = "http://www.netbian.com" text = start_request (url) parse (text) 1) # crawl the specified range page for i in range (2100): urlnum = url + "/ index_" + str (I) + ".htm" textnum = start_request (urlnum) parse (textnum,i) if _ _ name__ = "_ _ main__": main () this is the answer to the question about how Python crawls HD desktop wallpaper. I hope the above content can help you to a certain extent, if you still have a lot of doubts to be solved, you can follow the industry information channel to learn more related knowledge.
