Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "how to use Python to see what recent movies have just been released". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Project goal

Get details of the upcoming Cat's Eye movie.

Project preparation

Software: PyCharm

Required libraries: requests, lxml, random, time

Plug-in: Xpath

The website is as follows:

Https://maoyan.com/films?showType=2&offset={}

Click the button on the next page and observe the changes in the website as follows:

Https://maoyan.com/films?showType=2&offset=30https://maoyan.com/films?showType=2&offset=60https://maoyan.com/films?showType=2&offset=90

When you click on the next page, each additional page offset= () increases by 30, so you can use {} instead of the changed variable, and then use the for loop to traverse the URL to achieve multiple URL requests.

Project realization

1. Define a class class to inherit object, define init method to inherit self, and the main function main to inherit self. Import the required libraries and URLs, as shown below.

Import requestsfrom lxml import etreeimport timeimport randomclass MaoyanSpider (object): def _ _ init__ (self): self.url = "https://maoyan.com/films?showType=2&offset={}" def main (self): passif _ name__ = ='_ _ main__': spider = MaoyanSpider () spider.main ()

2. Randomly generate UserAgent.

For i in range (1,50): # ua.random, be sure to write here, each request will be randomly selected. Self.headers = {'User-Agent': ua.random,}

3. Send a request to get the page response.

Def get_page (self, url): # random.choice must be written here. Res = requests.get (url, headers=self.headers) res.encoding = 'utf-8' html = res.text self.parse_page (html) is randomly selected for each request.

4. Xpath parses the first-level page data and obtains the page information.

1) list of benchmark xpath node objects.

# create parsed object parse_html = etree.HTML (html) # benchmark xpath node object list dd_list = parse_html.xpath ('/ / dl [@ class= "movie-list"] / / dd')

2) traverse each node object in turn to extract data.

For dd in dd_list: name = dd.xpath ('. / / div [@ class= "movie-hover-title"] / / span [@ class= "name noscore"] / text ()') [0] .strip () star = dd.xpath ('. / / div [@ class= "movie-hover-info"] / div [@ class= "movie-hover-title"] [3] / text ()') [1] .strip () type = dd.xpath ('. / / div [@ class= "movie-hover-info"] / / div [@ class= "movie-hover-title"] [2] / text ()') [1] .strip () dowld=dd.xpath ('. / / div [@ class= "movie-item-hover"] / a Universe ref') [0] .strip () # print (movie_dict) movie =''[coming]

5. Define movie and save print data.

Movie ='[coming soon] Movie name:% s starring:% s Type:% s details link: https://maoyan.com%s=========================================================''% (name, star, type,dowld) print (movie)

6. Random.randint () method, which sets the time delay.

Time.sleep (random.randint (1,3))

7. Call the method to realize the function.

Html = self.get_page (url) self.parse_page (html)

Effect display

1. Click the small green triangle to run the input start page and end the page.

2. After running the program, the results are displayed in the console.

3. Click the blue download link to view the details on the network.

This is the end of the content of "how to use Python to see what movies have just been released recently". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.