How to realize the whole process of Python grabbing data to visualization
This article shows you how to achieve Python capture data to visualization of the whole process, the content is concise and easy to understand, absolutely can make your eyes bright, through the detailed introduction of this article, I hope you can get something.
1. Climb the target website: performance forecast _ data center _ flush finance and economics
(ps:headers will not be set up. You can read this article: Python uses requests.get to get the content of the web page is empty.')
Import pandas as pdimport numpy as npimport matplotlib.pyplot as pltimport reimport requests## guides headers= {'Accept':' text/html,*/* in advance of various packages that may be used. Qreply 0.01, 'hexin-v':' A2C11EVPFXAhNKl0PUpM6xp8MWU2aUUO5kMury YOdpwLngMrQpTgnkUwzZdaMpfilling, 'Referer':' http://data.10jqka.com.cn/financial/yjyg/', 'Cookie':' For 'User-Agent':', fill in'} url= "http://data.10jqka.com.cn/ajax/yjyg/date/2021-12-31/board/ALL/field/enddate/order/desc/page/{}/ajax/1/free/1/"result =''for i in range (1, 5): result + = requests.get (url.format (I), headers=headers) .textresult
Crawl succeeded, check the data:
2. Get the serial number, stock symbol, etc. Index = re.findall ('(. *)', result) index
148 pieces of data meet expectations, continue to look at the stock symbol (here use regular expression query, you can see this article explains regular expressions: regular expressions + common examples)
Stock_code = re.findall ('class= "stockCode" > (. *)', result) print (stock_code)
3. Composition DataFramedata = pd.DataFrame ({"stock_code": stock_code, "stock_name": stock_name, "performance": performance, "summary": summary}, index=index) data
Combination is successful
4. Processing data
Use regularization to filter out net profit and growth
Data ['start_profit'] = data [' summary'] .str.extract ('(- {0summary' 1}\ d {0 10}\ d {0 10}) data ['end_profit'] = data [' summary'] .str.extract ('to (- {0 1}\ d {0prit 10}\.) data ["start_range"] = data ['summary'] .str.extract (' (\ d {0prit 5})) \. {0summary' 1}\ d {0summary' 5})%') + "%" data ["end_range"] = data ['summary'] .str.extract (' to (\ d {0summary' 5})%') + "%" data.head ()
Judge profitability:
Def if_profit (x): if x in ['performance forecast decline', 'performance increase', 'performance decline', 'performance profit forecast', 'performance increase', 'expected turnaround']: return 1 if x in ['expected loss reduction', 'uncertainty', 'performance loss forecast', 'expected loss increase' 'expected continued loss']: return-1 return 0data ['is_profit'] = data.apply (lambda x: if_profit (x ["performance"]), axis=1) data.head () # # calculate profit yingli = (data [' is_profit'] = = 1). Sum () yingli## loss how much kuisun = (data ['is_profit'] =-1). Sum () kuisun
Visualization is achieved by drawing a pie chart:
Plt.pie ([yingli,kuisun], labels= ['profit', 'loss'], autopct='%.2f%%') font= {"family": "kaiti", "size": "15"} plt.rc ("font", * * font) plt.title
The above content is how to realize the whole process of Python grabbing data to visualization. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.
