How does Python crawl NBA tiger swoop player data

This article introduces Python how to crawl NBA tiger swoop player data, the content is very detailed, interested friends can refer to, hope to be helpful to you.

Tiger Pop is a serious and interesting community. Every day, many JRs share their views on basketball, football, game e-sports, sports equipment, film and television, car, digital, emotion and so on.

Victim address

Https://nba.hupu.com/stats/players

Knowledge points of this article:

Systematic analysis of the nature of web pages

Structured data parsing

Csv data saving

Environment introduction:

Python 3.6

Pycharm

Requests

Csv

General steps for crawler cases

1. Make sure the url address (web page analysis) is half done.

two。 Send a network request requests (js\ html\ css)

3. Data parsing (filtering data)

4. Save data (local file\ database)

Partial code

Import tool

Import requests # third-party tool import parsel # data parsing tool (css\ regular expression\ xpath) import csv

Make sure the url address (web page analysis) is half done (static page / dynamic page)

Url = 'https://nba.hupu.com/stats/players/pts/{}'.format(page)

Send a network request requests (js\ html\ css)

Response = requests.get (url=url) html_data = response.text

Data parsing (filtering data)

Selector = parsel.Selector (html_data) trs = selector.xpath ('/ / tbody/tr [not (@ class= "color_font1 bg_a")]') for tr in trs: rank = tr.xpath ('. / td [1] / text ()'). Get () # ranking player = tr.xpath ('. / td [2] / a/text ()'). Get () # player team = Tr.xpath ('. / td [3] / a/text ()). Get () # team score = tr.xpath ('. / td [4] / text ()'). Get () # score hit_shot = tr.xpath ('. / td [5] / text ()'). Get () # hit-shot hit_rate = tr.xpath ('. / td [6] / text ()) '). Get () # hit rate hit_three = tr.xpath ('. / td [7] / text ()'). Get () # hit-three-point three_rate = tr.xpath ('. / td [8] / text ()'). Get () # three-point hit rate hit_penalty = tr.xpath ('. / td [9] / text ()'). Get () # hit -penalty_rate = tr.xpath ('. / td [10] / text ()'). Get () # hit percentage of free throws session = tr.xpath ('. / td [11] / text ()'). Get () # playing_time = tr.xpath ('. / td [12] / text ()'). Get () # playing time print (rank Player, team, score, hit_shot, hit_rate, hit_three, three_rate, hit_penalty, penalty_rate, session, playing_time) data_dict = {'Rank': rank, 'player': player, 'team': team, 'score': score, 'hit-shot': hit_shot, 'hit percentage': hit_rate, 'hit-three points': hit_three '3-point shooting percentage': three_rate, 'shooting-free throw': hit_penalty, 'free throw shooting percentage': penalty_rate, 'number of games': session 'playing time': playing_time} csv_write.writerow (data_dict) # students who want the complete source code can follow my official account: squirrels love cookies # reply to "Tiger Pop NBA" and get it for free

Run the code and the effect is as follows

About Python how to crawl NBA tiger swoop player data is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.