Shulou(Shulou.com)06/02 Report--

This article mainly introduces how to use the Python Dash library to make a cool visualization screen, with a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Introduction

Through Python's Dash library, to create a cool visual screen!

First, let's take a look at the overall effect. It seems pretty good.

Mainly use Python's Dash library, Plotly library, Requests library.

Among them, Requests crawls data, Plotly makes visual charts, and Dash builds visual pages.

The original data is Xiao F's blog data, which is stored in the MySqL database.

From this point of view, it is similar to the process of building a Streamlit library.

About the Dash library, there is not much information on the Internet, basically can only see the official documents and cases, the following is a brief introduction of Xiao F.

Dash is an efficient Python framework for building Web applications, especially for people who use Python for data analysis.

Dash is built on Flask,Plotly.js and React.js and is very suitable for building data visualization applications using a highly customized user interface in pure Python.

Related documents

Description: https://dash.plotly.com/introduction

Case: https://dash.gallery/Portal/

Source code: https://github.com/plotly/dash-sample-apps/

Specifically, we can look at the documents to learn, more hands-on practice.

Let's show you how to build a visual screen through Dash.

data

The data used is blog data, mainly the information of the two red boxes below.

Crawled through the crawler code and stored in the MySQL database.

Of which the installation of MySQL, we can do their own Baidu, are very simple.

After installation, enable and create the database.

# launch MySQL and enter the password mysql-u root-p # to create a database named my_database create database my_database

Other related operation commands are shown below.

# display all databases in MySQL show databases; # Select my_database database use my_database; # display all tables in my_database database show tables; # delete table drop table info;drop table `2021-12-26`; # display the contents of the table and execute the SQL query statement select * from info;select * from `2021-12-26`

After completing the above steps, you can run the crawler code.

The data crawling code is as follows. The library pymysql is used here, which needs to be installed by pip.

Import requestsimport refrom bs4 import BeautifulSoupimport timeimport randomimport pandas as pdfrom sqlalchemy import create_engineimport datetime as dt def get_info (): "get the first column of information data on the big screen" headers = {'User-Agent':' Mozilla/5.0 (MSIE 10.0; Windows NT 6.1) " Trident/5.0)', 'referer':' https: / / passport.csdn.net / login',} # my blog address url = 'https://blog.csdn.net/river_star1/article/details/121463591' try: resp = requests.get (url, headers=headers) now = dt.datetime.now () .strftime ("% Y-%m-%d% X") soup = BeautifulSoup (resp.text 'lxml') author_name = soup.find (' div', class_='user-info d-flex flex-column profile-intro-name-box'). Find ('a'). Get_text (strip=True) head_img = soup.find ('div', class_='avatar-box d-flex justify-content-center flex-column'). Find (' a'). Find ('img') [' src'] row1_nums = soup.find_all ('div') Class_='data-info d-flex item-tiling') [0] .find _ all ('span', class_='count') row2_nums = soup.find_all (' div', class_='data-info d-flex item-tiling') [1]. Find _ all ('span', class_='count') level_mes = soup.find_all (' div' Class_='data-info d-flex item-tiling') [0] .find _ all ('dl') [- 1] [' title'] .split (',') [0] rank = soup.find ('div', class_='data-info d-flex item-tiling'). Find_all (' dl') [- 1] ['title'] info = {' date': now,# time 'head_img': head_img # profile picture 'author_name': author_name,# user name' article_num': str (row1_nums [0] .get _ text ()), # number of articles' fans_num': str (row2_nums [1] .get _ text ()), # number of fans' like_num': str (row2_nums [2] .get _ text ()) # likes' comment_num': str (row2_nums [3] .get _ text ()), # comments' level': level_mes,# rating 'visit_num': str (row1_nums [3] .get _ text ()), # visits' score': str (row2_nums [0] .get _ text ()) # integral 'rank': str (row1_nums [2] .get _ text ()), # ranking} df_info = pd.DataFrame ([info.values ()] Columns=info.keys () return df_info except Exception as e: print (e) return get_info () def get_type (title): "set article type (based on article name)" the_type = 'other' article_types = ['Project', 'data Visualization', 'Code', 'Chart', 'Python',' Visualization', 'data' 'interview', 'video', 'dynamic', 'download'] for article_type in article_types: if article_type in title: the_type = article_type break return the_type def get_blog (): "get the second and third columns of information data on the big screen" headers = {'User-Agent':' Mozilla/5.0 (MSIE 10.0) " Windows NT 6.1; Trident/5.0), 'referer':' https: / / passport.csdn.net / login',} base_url = 'https://blog.csdn.net/river_star1/article/list/' resp = requests.get (base_url+ "1", headers=headers, timeout=3) max_page = int (re.findall (r'var listTotal = (\ d +)) ', resp.text) [0]) / / 40 resp.text 1 df = pd.DataFrame (columns= [' url', 'title',' date', 'read_num',' comment_num', 'type']) count = 0 for i in range (1, max_page+1): url = base_url + str (I) resp = requests.get (url, headers=headers) soup = BeautifulSoup (resp.text 'lxml') articles = soup.find ("div", class_='article-list'). Find_all (' div' Class_='article-item-box csdn-tracking-statistics') for article in articles [1:]: a_url = article.find ('h5'). Find (' a') ['href'] title = article.find (' h5'). Find ('a'). Get_text (strip=True) [2:] issuing_time = article.find ('span'') Class_= "date") .get_text (strip=True) num_list = article.find_all ('span') Class_= "read-num") read_num = num_list [0] .get _ text (strip=True) if len (num_list) > 1: comment_num = num_list [1] .get _ text (strip=True) else: comment_num = 0 the_type = get_type (title) df.loc [count] = [a_url, title Issuing_time, int (read_num), int (comment_num), the_type] count + = 1 time.sleep (random.choice )) return df if _ _ name__ ='_ main__': # Today's time today = dt.datetime.today () .strftime ("% Y-%m-%d") # Connect to the mysql database engine = create_engine ('mysql+pymysql://root:123456@localhost/my_database?charset=utf8') # get the first column of information data on the large screen and write it to the info table of the my_database database If the table already exists, delete and overwrite df_info = get_info () print (df_info) df_info.to_sql ("info", con=engine, if_exists='replace', index=False) # to get the second and third columns of information on the large screen, and write it to the date table of the my_database database, if the table already exists. Delete overlay df_article = get_blog () print (df_article) df_article.to_sql (today, con=engine, if_exists='replace', index=True)

After running successfully, you can go to the database to query the information.

Info table, including date, headpicture, blog name, number of articles, number of fans, number of likes, number of comments, number of grades, number of visits, product score, ranking number.

Date table, including article address, title, date, number of readings, number of comments, type.

The crawler code can be set to run regularly. The info table is 60 seconds and the date table is 60 minutes.

Try not to be too frequent, easy to be blocked by IP, or choose to use proxy pools.

In this way, data can be updated in real time.

Now that you have the data, you can write the page.

Large screen construction

Import the relevant Python libraries, which can also be installed through pip.

From spider_py import get_info, get_blogfrom dash import dccimport dashfrom dash import htmlimport pandas as pdimport plotly.graph_objs as gofrom dash.dependencies import Input, Outputimport datetime as dtfrom sqlalchemy import create_enginefrom flask_caching import Cacheimport numpy as np

Set some basic configuration parameters, such as database connection, web page style, Dash instance, chart color.

# Today's time today = dt.datetime.today (). Strftime ("% Y-%m-%d") # Connect database engine = create_engine ('mysql+pymysql://root:123456@localhost/my_database?charset=utf8') # Import css style external_css = ["https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css"," "https://cdnjs.cloudflare.com/ajax/libs/skeleton/2.0.4/skeleton.min.css"] # create an instance app = dash.Dash (_ _ name__, external_stylesheets=external_css) server = app.server # you can choose to use cache to reduce frequent data requests # cache = Cache (app.server, config= {# 'CACHE_TYPE':' filesystem' # 'CACHE_DIR':' cache-directory'#}) # read the data from the info table info = pd.read_sql ('info', con=engine) # chart color color_scale = [' # 2c0772colors,'# 3d208ewings,'# 8D7DFFF bands,'# CDCCFF','# C7FFFBPs,'# ff2c6d','# 564b43colors,'# 161d33']

The cache code is commented out here, and if there are frequent page refresh requests, you can choose to use it.

Def indicator (text, id_value): "first column text and numeric information display" return html.Div ([html.P (text, className= "twelve columns indicator_text"), html.P (id=id_value, className= "indicator_value"),], className= "col indicator") def get_news_table (data): "" get the article list According to the reading order "" df = data.copy () df.sort_values ('read_num', inplace=True, ascending=False) titles = df [' title'] .tolist () urls = df ['url'] .tolist () return html.Table ([html.Tbody ([html.Td (html.A (titles [I], href=urls [I], target= "_ blank") ), style= {'height':' 30pxspell, 'fontSize':' 16'}) for i in range (min (len (df), 100)])], style= {"height": "width": "98"}) # @ cache.memoize (timeout=3590), optionally set cache I didn't use def get_df (): "get the latest article data of the day" df = pd.read_sql (today Con=engine) df ['date_day'] = df [' date'] .apply (lambda x: x.split (') [0]). Astype ('datetime64 [ns]') df ['date_month'] = df [' date'] .apply (lambda x: X [: 7] .split ('-') [0] + "year" + x [: 7] .split ('-') [1] + "month") df ['weekday'] = Df ['date_day']. Dt.weekday df [' year'] = df ['date_day']. Dt.year df [' month'] = df ['date_day']. Dt.month df [' week'] = df ['date_day'] .dt.isocalendar (). Week return df # picture and title head = html.Div ([html.Div (html.Img (src='./assets/img.jpg')) Height=, style= {"float": "left", "height": "90%", "margin-top": "5px", "border-radius": "50%", "overflow": "hidden"}), html.Span ("{} blog Dashboard" .format (info ['author_name'] [0]), className='app-title'),] ClassName= "row header") # text and numeric information of the first column columns = info.columns [3:] col_name = ['number of articles', 'number of followers', 'number of likes', 'number of comments', 'rating', 'number of visits', 'points', 'ranking'] row1 = html.Div ([indicator (col_name [I], col) for I, col in enumerate (columns)] ClassName='row') # second column row2 = html.Div ([html.Div ([html.P ("monthly article writing"), dcc.Graph (id= "bar", style= {"height": "90%", "width": "98%"}, config=dict (displayModeBar=False),)], className= "col-4 chart_div",), html.Div ([html.P ("proportion of various types of articles") Dcc.Graph (id= "pie", style= {"height": "90%", "width": "98%"}, config=dict (displayModeBar=False),)], className= "col-4 chart_div"), html.Div ([html.P ("reading of various types of articles"), dcc.Graph (id= "mix", style= {"height": "90%", "width": "98%"}, config=dict (displayModeBar=False)) ), className= "col-4 chart_div",)], className='row') # statistics, mine is 2019 2020 2021years = get_df () ['year'] .unique () select_list = [' monthly article', 'type percentage', 'type readings', 'daily situation'] # two interactive drop-down options dropDowm1 = html.Div ([dcc.Dropdown (id='dropdown1') Options= [{'label':' {} year '.format (year),' value': year} for year in years], value=years [1], style= {'width':' 40%'})], className='col-6', style= {'padding':' 2pxspell, 'margin':' 0px 5px 0px'}), html.Div ([dcc.Dropdown (id='dropdown2') Options= [{'label': select_list [I],' value': item} for I, item in enumerate (['bar',' pie', 'mix',' heatmap'])], value='heatmap', style= {'width':' 40%'})], className='col-6', style= {'padding':' 2pxcodes, 'margin':' 0px 5px 0px'})] ClassName='row') # third column row3 = html.Div ([html.Div ([html.P ("Daily Writing"), dcc.Graph (id= "heatmap", style= {"height": "90%", "width": "98%"}, config=dict (displayModeBar=False),)], className= "col-6 chart_div",), html.Div ([html.P ("list of articles") Html.Div (get_news_table (get_df ()), id='click-data'),], className= "col-6 chart_div", style= {"overflowY": "scroll"})], className='row') # overall situation app.layout = html.Div ([# timer dcc.Interval (id= "stream", interval=1000*60, n_intervals=0), dcc.Interval (id= "river", interval=1000*60*60, n_intervals=0) Html.Div (id= "load_info", style= {"display": "none"},), html.Div (id= "load_click_data", style= {"display": "none"},), head, html.Div ([row1, row2, dropDowm1, row3,], style= {'margin':' 0% 30px'}),]))

The above code, which is the layout of the web page, has the following effect.

A web page can be divided into three columns. The first is the data presentation in the info table, and the second and the third are the data presentation of blog posts.

The relevant data needs to be updated through the callback function so that it can be refreshed in real time.

The callback function code for each value and chart is shown below.

# callback function, refresh info data in 60 seconds, that is, the values of the first column are refreshed in real time @ app.callback (Output ('load_info',' children'), [Input ("stream", "n_intervals")]) def load_info (n): try: df = pd.read_sql ('info', con=engine) return df.to_json () except: pass # callback function, refresh today's data in 60 minutes That is, the values of the second and third columns are refreshed in real time (crawling the article data and writing it to the database) @ app.callback (Output ('load_click_data',' children'), [Input ("river", "n_intervals")]) def cwarl_data (n): if n! = 0: df_article = get_blog () df_article.to_sql (today, con=engine, if_exists='replace', index=True) # callback function The first bar chart @ app.callback (Output ('bar',' figure'), [Input ("river", "n_intervals")]) def get_bar (n): df = get_df () df_date_month = pd.DataFrame (df ['date_month']. Value_counts (sort=False)) df_date_month.sort_index (inplace=True) trace = go.Bar (x=df_date_month.index) Y=df_date_month ['date_month'], text=df_date_month [' date_month'], textposition='auto', marker=dict (color='#33ffe6')) layout = go.Layout (margin=dict (gridcolor='#e2e2e2'), yaxis=dict (gridcolor='#e2e2e2'), paper_bgcolor='rgba)', plot_bgcolor='rgba' ) return go.Figure (data= [trace], layout=layout) # callback function, the middle pie chart @ app.callback (Output ('pie',' figure'), [Input ("river", "n_intervals")]) def get_pie (n): df = get_df () df_types = pd.DataFrame (df ['type']. Value_counts (sort=False) trace = go.Pie (labels=df_types.index) Values=df_types ['type'], marker=dict (colors=color_scale [: len (df_types.index)]) layout= go.Layout (margin=dict (lager 50, raster 50, tweak 50, baud 50), paper_bgcolor='rgba (0trace 0)', plot_bgcolor='rgba',) return go.Figure (data= [trace], layout=layout) # callback function Lower left corner thermal map @ app.callback (Output ('heatmap',' figure'), [Input ("dropdown1", "value"), Input ('river',' nasty intervals')]) def get_heatmap (value, n): df = get_df () grouped_by_year = df.groupby ('year') data = grouped_by_year.get_group (value) cross = pd.crosstab (data [' weekday'] Data ['week']) cross.sort_index (inplace=True) trace = go.Heatmap (x = [' {} week '.format (I) for i in cross.columns], y = ["week {}" .format (item1) if I! = 6 else "Sunday" for i in cross.index], z=cross.values, colorscale= "Blues", reversescale=False, xgap=4, ygap=5 Showscale=False) layout= go.Layout (margin=dict (lumped 50, rang 40, tweak 30, baked 50),) return go.Figure (data= [trace], layout=layout) # callback function, second histogram (histogram + line chart) @ app.callback (Output ('mix',' figure'), [Input ("river") "n_intervals")]) def get_mix (n): df = get_df () df_type_visit_sum = pd.DataFrame (df ['read_num'] .groupby (df [' type']) .sum () df ['read_num'] = df [' read_num'] .astype ('float') df_type_visit_mean = pd.DataFrame (df [' read_num'] .groupby (df ['type']) ). Agg ('mean') .round (2) trace1 = go.Bar (x=df_type_visit_sum.index Y=df_type_visit_sum ['read_num'], name=' Total Reading', marker=dict (color='#ffc97b'), yaxis='y',) trace2 = go.Scatter (x=df_type_visit_mean.index, y=df_type_visit_mean ['read_num'], name=' average Reading', yaxis='y2' Line=dict (color='#161D33')) layout = go.Layout (margin=dict (lager 60, rpm 60, tweak 30, bounce 50), showlegend=False, yaxis=dict (total side='left', title=' readings', gridcolor='#e2e2e2'), yaxis2=dict (showgrid=False, # grid title=' reading average' Anchor='x', overlaying='y', side='right'), paper_bgcolor='rgba', plot_bgcolor='rgba',) return go.Figure (data= [trace1, trace2], layout=layout) # Click the event, select the two drop-down options, and click on the chart in the corresponding area. The list of articles will be refreshed @ app.callback (Output ('click-data',' children'), [Input ('pie',' clickData'), Input ('bar',' clickData'), Input ('mix',' clickData'), Input ('heatmap',' clickData'), Input ('dropdown1',' value'), Input ('dropdown2',' value') ]) def display_click_data (pie, bar, mix, heatmap, d_value Fig_type): try: df = get_df () if fig_type = 'pie': type_value = pie [' points'] [0] ['label'] # date_month_value = clickdata [' points'] [0] ['x'] data = df [df ['type'] = = type_value] elif fig_type = =' bar': Date_month_value = bar ['points'] [0] [' x'] data = df [df ['date_month'] = = date_month_value] elif fig_type = =' mix': type_value = mix ['points'] [0] [' x'] data = df [df ['type'] = = type_value] else: Z = heatmap ['points'] [0] [' z'] if z = 0: return None else: week = heatmap ['points'] [0] [' x'] [1 if 1] weekday = heatmap ['points'] [0] [' y'] [- 1] if weekday = = 'Day': Weekday = 7 year = d_value data = df [(df ['weekday'] = = int (weekday)-1) & (df [' week'] = = int (week)) & (df ['year'] = = year)] return get_news_table (data) except: numerical def update_info (col) of return None # first column: def get_data (json N): df = pd.read_json (json) return df [col] [0] return get_data for col in columns: app.callback (Output (col, "children"), [Input ('load_info',' children'), Input ("stream", "n_intervals")]) (update_info (col))

The data and style of the chart are all set here, and the data interaction between the two drop-down bars is also completed here.

You need to pay attention to the type of drop-down bar on the right, which is the same as the type of chart you want to click on, so that the list of articles will be updated.

The daily situation corresponds to the thermal map, the type reading volume corresponds to the second column and the third chart, the type proportion corresponds to the pie chart, and the monthly article corresponds to the click event of the first bar chart.

Finally, start the program code.

If _ _ name__ ='_ _ main__': # debug mode, port 7777 app.run_server (debug=True, threaded=True, port=7777) # normal mode, the debug button in the lower right corner of the page will not appear # app.run_server (port=7777)

In this way, you can see the large visual screen page locally, and the browser opens the following address.

Http://127.0.0.1:7777

The layout and background color of the web page are mainly set by CSS.

This part may take time to understand.

Body {margin:0; padding: 0; background-color: # 161D33; font-family: 'Open Sans', sans-serif; color: # 506784;-webkit-user-select: none; / * Chrome all / Safari all * /-moz-user-select: none; / * Firefox all * /-ms-user-select: none; / * IE 10 + * / user-select: none / * Likely future * /} .modal {display: block; / * Hidden by default * / position: fixed; / * Stay in place * / z-index: 1000; / * Sit on top * / left: 0; top: 0; width: 100%; / * Full width * / height: 100%; / * Full height * / overflow: auto; / * Enable scroll if needed * / background-color: rgb / * Fallback color * / background-color: rgba; / * Black w / opacity * /}. Modal-content {background-color: white; margin: 5% auto; / * 15% from the top and centered * / padding: 20px; width: 30%; / * Could be more or less, depending on screen size * / color:#506784;}. _ dash-undo-redo {display: none;}. App-title {color:white Font-size:3rem; letter-spacing:-.1rem; padding:10px; vertical-align:middle} .header {margin:0px; background-color:#161D33; height:70px; color:white; padding-right:2%; padding-left:2%} .header {border-radius: 5px; background-color:# f9f9f9; margin: 10px; padding: 15px; position: relative; box-shadow: 2px 2px 2px lightgrey Value _ value {text-align:center; color: # 2a3f5f; font-size: 35px;} .add {height: 34px; background: # 119DFF; border: 1px solid # 119DFF; color: white;} .chart _ div {background-color: # f9f9f9; border-radius: 5px; height: 390px Margin:5px; padding: 15px; position: relative; box-shadow: 2px 2px 2px lightgrey;} .col-4 {flex: 032.65%; max-width: 33%;} .col-6 {flex: 0.049.3%; max-width: 50%;} .chart _ div p {color: # 2a3f5f; font-size: 15px; text-align: center;} td {text-align: left; padding: 0px } table {border: 1px; font-size:1.3rem; width:100%; font-family:Ubuntu;} .tabs _ div {margin:0px; height:30px; font-size:13px; margin-top:1px} tr:nth-child (even) {background-color: # d6e4ea;-webkit-print-color-adjust: exact } Thank you for reading this article carefully. I hope the article "how to use Python Dash Library to make a cool visual screen" shared by the editor will be helpful to everyone. At the same time, I also hope that you will support and follow the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.