Shulou(Shulou.com)06/03 Report--

This article mainly explains "what is the difference between urllib and requests in Python crawler". The content in the article is simple and clear, and it is easy to learn and understand. Please follow Xiaobian's train of thought to study and learn "what is the difference between urllib and requests in Python crawler"?

1. The first step of obtaining web page data is to introduce the module.

It is obvious that the modules introduced by the two are different.

The second step is a simple request initiated by a web page.

Urllib acquires data through the urlopen method.

Requests needs to get the data through the response type of the web page.

The third step is data encapsulation.

For complex data requests, it is certainly not possible for us to simply use the urlopen method. Finally, if your time is not very tight, and want to improve quickly, the most important thing is not afraid of hardship, it is suggested that you can contact dimension: 762459510, that is really good, many people are making rapid progress, you are not afraid of hardship! You can add it and have a look at it.

In urllib, we know that for sites with an anti-crawler mechanism, we need to encapsulate the URL to get the data. We can review the contents of the previous lessons:

In the requests module, there is no need for such a complex operation. Just add the parameter headers in the second step:

2. Parsing web page data

Both urllib and requests can parse the data through bs4 and re, and requests can also parse through xpath. The specific analytical method will be explained in detail later.

3. Save data

Urllib needs to introduce xlwt module to create new tables and write data in sheet tables. Finally, if your time is not very tight, and want to improve quickly, the most important thing is not afraid of hardship, it is suggested that you can contact dimension: 762459510, that is really good, many people are making rapid progress, you are not afraid of hardship! You can add it and have a look at it.

Requests through with... As directly impresses the data:

Thank you for your reading, the above is the content of "what is the difference between urllib and requests in Python crawler". After the study of this article, I believe you have a deeper understanding of what is the difference between urllib and requests in Python crawler. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.