Shulou(Shulou.com)06/03 Report--

Editor to share with you how to deal with page turning in python scrapy. I hope you will get something after reading this article. Let's discuss it together.

Description

1. According to the running characteristics of the scrapy framework, page flipping must be separated from processing data, otherwise it cannot be performed.

2. Determine the url address, construct the request object of the url address, and give the request to the engine.

Construction request

Scrapy.Request (url, callback)

Callback: specifies the name of the parsing function, indicating which function is used to parse the response returned by the request.

Example

Scrapy.Request (url, callback=None, method='GET', headers=None, body=None, cookies=None, meta=None, encoding='utf-8', priority=0, dont_filter=False, errback=None, flags=None) after reading this article, I believe you have a certain understanding of "how to deal with page turning in python". If you want to know more about it, welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.