Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to use Python to get the popularity list of anchors". Interested friends might as well take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to use Python to get the popularity list of anchors.

Knowledge points of this article:

1. Systematic web page analysis

2. Data processing

3. The use of jsonpath module

Environment introduction:

Python 3.6

Pycharm

Requests

Josnpath

General ideas of reptiles

1. Analyze the target web page and determine the crawled url path and headers parameters.

2. Send a request-- requests simulates the browser to send a request to obtain response data

3. Parsing data-jsonpath module: JsonPath is an information extraction class library and a tool for extracting specified information from JSON documents.

4. Data output

Crawl data

1. Import related databases

Import jsonpathimport requestsimport pprint

2. Analyze the target web page to determine the crawled url path and headers parameters

URL address:

Base_url = 'https://www.douyu.com/gapi/rknc/directory/yzRec/1'headers = {' User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36'}

3. Send a request-- requests simulates the browser to send a request to obtain response data

Response = requests.get (url=base_url, headers=headers) json_data = response.json ()

4. Parsing data-- jsonpath module: JsonPath is an information extraction class library and a tool for extracting specified information from JSON documents.

Dictionaries are arranged in descending order according to values.

Sorted method, three parameters of sorted:

Iterable: iterable object.

Key: an element that is mainly used for comparison, with only one parameter. The specific parameter of the function is taken from the iterable object. You can specify an element in the iterable object to sort.

Reverse: collation, reverse = True descending, reverse = False ascending (default).

Names = jsonpath.jsonpath (json_data,'$.. nn') # $extract from the root node.. Extract the hot = jsonpath.jsonpath (json_data,'$.. ol') print (names) print (hot) item_dict = {} for name, value in zip (names, hot): # print (name, value) item_ strings [name] = valueprint (item_dict) print (item_dict.items ()) # items () function across nodes to return a traverable (key, value) tuple in a list. Change_score = sorted (item_dict.items (), key=lambda x: X [1], reverse=True) # lambda 1 is the index print (change_score) of the tuple

5. Data output

Enumerate enumeration enumerate is mostly used to get the count in the for loop, and it can be used to obtain both the index and the value, that is, you can use enumerate when you need index and value values.

Or player, hot_num in enumerate (change_score): print ('No. {} name = yes {} = popularity value is: {}' .format (player) so far, I believe you have a better understanding of "how to use Python to obtain the popularity list of anchors". You might as well do it in practice! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.