Shulou(Shulou.com)06/03 Report--

The main content of this article is to explain the realization of the introduction case of Python crawler, enter the desktop wallpaper net beauty picture collection, interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "Python crawler introduction case of the realization of enter desktop wallpaper net beauty picture collection" bar!

Catalogue

Knowledge point

Environment

Target URL:

Crawler code

Import module

Send a network request

Get the source code of the web page

Extract the link address of the details page of each album

Replace all the picture links with larger ones

Save picture name

Turn the page

Crawl result

Knowledge point

Requests

Parsel

Re

Os

Environment

Python3.8

Pycharm2021

Target URL:

Https://mm.enterdesk.com/bizhi/63899-347866.html

[full version of paid VIP] A course that you can learn as long as you read it, 80 episodes of Python basic introductory video teaching

Click here to watch it online for free.

Note: when we view the source code of the web page (1. The console shall prevail 2. Right-click to view the web page source code 3. Elements panel)

Send a network request

Get the source code of the web page

Extract the desired picture link css style extract xpath re regular expression bs4

Replace all the picture links with larger ones

Save pictures

Crawler code import module import requests # third party library pip install requestsimport parsel # third party library pip install parselimport os # New folder send network request response = requests.get ('https://mm.enterdesk.com/bizhi/64011-348522.html') get web source code data_html = response_1.text extract the details page link address of each album selector_1 = parsel.Selector (data_html) ) photo_url_list = selector_1.css ('.egeli _ pic_dl dd a::attr (href)') .getall () title_list = selector_1.css ('.egeli _ pic_dl dd an img::attr (title)') .getall () for photo_url Title in zip (photo_url_list, title_list): print (Flying * crawling {title} *') response = requests.get (photo_url) #: request successful ID selector = parsel.Selector (response.text) # extract the desired image link [first link, second link ....] Img_src_list = selector.css ('.swiper-wrapper an img::attr (src)'). Getall () # create a new folder if not os.path.exists ('img/' + title): os.mkdir (' img/' + title) replace all the picture links with larger for img_src in img_src_list: # string replacement img_url = img_src.replace ('_ 360 '_ source') save picture name # picture audio and video binary data contentimg_data = requests.get (img_url). Content# picture name string segmentation will return us a list img_title = img_url.split (' /') [- 1] with open (f'img/ {title} / {img_title}') Mode='wb') as f: f.write (img_data) print (img_title, 'saved successfully!') Turn the page page_html = requests.get ('https://mm.enterdesk.com/').textcounts = parsel.Selector (page_html) .css (' .wrap.no _ a::attr (href)'). Get (). Split ('/') [- 1] .split ('.) [0] for page in range (1 Int (counts) + 1): print (faded page-crawling page {page} -') sends a network request response_1 = requests.get (f'https:/) / mm.enterdesk.com/ {page} .html') crawl result

At this point, I believe that everyone on the "Python crawler introduction case of the realization of enter desktop wallpaper network beauty picture collection" have a deeper understanding, might as well to the actual operation of it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.