Shulou(Shulou.com)06/03 Report--

Editor to share with you how to simulate login in scrapy in python, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1. Requests module. Directly carry the cookies request page.

Find the url and send a post request to store the cookie.

2. Selenium (the browser automatically handles cookie).

Find the appropriate input tag, enter the text, and click Log in.

3. Scrapy takes cookies directly.

Find the url and send a post request to store the cookie.

#-*-coding: utf-8-*-import scrapyimport re class GithubLoginSpider (scrapy.Spider): name = 'github_login' allowed_domains = [' github.com'] start_urls = ['https://github.com/login'] def parse (self) Response): # send a Post request to get Cookies authenticity_token = response.xpath ('/ / input [@ name= "authenticity_token"] / @ value'). Extract_first () utf8 = response.xpath ('/ / input [@ name= "utf8"] / @ value'). Extract_first () commit = response.xpath ('/ / input [@ name= "commit"] / @ value') .extract_first () form_data = {'login':' pengjunlee@163.com' 'password':' 123456, 'webauthn-support':' supported', 'authenticity_token': authenticity_token,' utf8': utf8, 'commit': commit} yield scrapy.FormRequest ("https://github.com/session", formdata=form_data, callback=self.after_login) def after_login (self Response): # verify whether the request is successful print (re.findall ('Learn Git and GitHub without any codes, response.body.decode () these are all the contents of the article "how to simulate login in scrapy in python" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.