Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you about how to climb the movie paradise data in Python. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

First open Pycharm, click File, and then click setting.

When it opens, this interface will appear and click on your project name (project: (your project name)) project interpreter and click the plus sign to download the library we need (requests,requests,time,re module), as shown in the following figure.

[III. Project implementation]

We need (requests,requests,time,re module), as shown in the following figure.

This time is used to prevent anti-crawling, setting the time delay.

First of all, let's analyze the characteristics of the next page of this website.

Use the for loop to traverse the URL in the main method main function.

It means you're half done. Come on!

Now we need to make requests for these URLs, and in order to see it more intuitively, we use a class to write.

We use requests to request that the website's code is gbk (what do you think of the site's code?).

Open a website to right-check the tag in header. Take this site as an example, you can see charset= "gb312".

This gb2312 is coding. There are two common ways of coding (utf_8, gbk).

We can verify that the request has been made. Using Print (html) to see this result (a complete html page) indicates that the request was successful.

So we can first find the table, layer by layer to find, you can refer to the following figure.

Click on the second-level page as shown in the figure and right-click the download link, as shown below:

We use regular expression parsing to get our download link address, as shown in the following figure:

Get the result, as shown in the following figure:

Finally, let's optimize the requested code. It's a little repetitive. Let's optimize it.

After using a value to save the contents of the request header, we can only call this method to make the request, as shown in the following figure:

The above is how to crawl the movie paradise data in the Python shared by the editor. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.