Shulou(Shulou.com)06/01 Report--

This article mainly introduces how python crawls wallpaper website data, which has certain reference value. Interested friends can refer to it. I hope you will gain a lot after reading this article. Let Xiaobian take you to understand it together.

preface

The text and pictures of this article come from the network, only for learning and communication, and do not have any commercial use. The copyright belongs to the original author. If you have any questions, please contact us in time for processing.

Recently found another great wallpaper site

environment

Python3.6

pycharm

This objective

Crawl website HD

website address

https://wallhaven.cc/Crawler Code

import tool

import requestsimport re

the request to the web

url = 'https://wallhaven.cc/toplist? page={}'.format(page)headers = { 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'}response = requests.get(url=url, headers=headers)

Parsing website data

for i in urls: response_2 = requests.get(url=i, headers=headers) img_url = re.findall('

save data

def download(title, url): path = 'Save address' + title response = requests.get(url=url) with open(path, mode='wb') as f: f.write(response.content) Thank you for reading this article carefully, I hope Xiaobian shared "python how to crawl wallpaper website data" This article is helpful to everyone, but also hope that everyone supports more, pay attention to the industry information channel, more relevant knowledge waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.