Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to use the requests module in python. Xiaobian thinks it is quite practical, so share it with you as a reference. I hope you can gain something after reading this article.

Requests module is a module for network requests, mainly used to simulate browser requests. In fact, there are many similar modules, such as urllib, urllib2, httplib, httplib2, they basically provide similar functions. But these modules are complex and almost obsolete, and the requests module is simple, powerful and efficient, making it irrelevant among the many network request modules.

requests used

Environment installation: pip install requests

Use process:

specified URL

Send requests based on requests module

Gets the data value in the response object

Persistent storage (not required)

Case: Crawling Baidu home page data

#1. import requests#2. Specify urlurl = "https://www.baidu.com"#3. Send the request using the GET method, which returns a response object response = requests.get(url=url)#4. Get response data print(response.status_code) #Print status code print(response.url) #print request urlprint(response.headers) #print response header information print(response.text) #Print source code as text #Save data response.encoding = 'utf-8' #Specify encoding format, otherwise open garbled text = response.textwith open ('./ 2.html','w',encoding='utf-8') as f: f.write(text)

Resolves the Chinese garbled code problem of the data requested by requests--"Requests return content Chinese garbled code problem

requests Request method

The above example requests send a GET request method, but there are other request methods besides that. The most common methods are GET and POST.

res = requests.get ()

res = requests.post ()

res = requests.put ()

res = requests.delete ()

res = requests.head ()

res = requests.options ()

And when specifying a method to send a request, sometimes you need to specify some parameters in requests.get(url=url, xx = xx) in the request method brackets, as follows. first understand

method

parameter name

HTTP header

headers

GET parameter

params

POST parameters

data

file

files

Cookies

cookies

redirection processing

allow_ redirects = False/True

timeout

timeout

certificate verification

verify = False/True

Workflow (delayed download)

stream=False/ True

event hooks

hooks=dict(response=)

authentication

auth=

agent

proxies=

requests Response object properties

When crawling Baidu homepage above, response = requests.get(url=url) returns a response object, and if we want to obtain specific data such as response code or web page source code, we need to specify the attributes of the response object to obtain. For example, response.status_code gets the response code

Get request url res. url

status code res. status_code

Response data (in string form) res . text

Return is a native string, bytes type res. content

View server response headers res. headers

View cookies res.cookies

About "how to use the requests module in python" this article is shared here, I hope the above content can be of some help to everyone, so that you can learn more knowledge, if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.