Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "how to use python data visualization Bokeh and Bottle.py to display your data on the web page". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Catalogue

1. Focus of the article and introduction of the project

two。 Dataset research and chart preparation

2.1 Import dataset

2.2 drawing a chart

Chart 1: daily AQI change curve of Shanghai, Beijing and Shenzhen in 2019

Chart 2: comparison of monthly average AQI among Shanghai, Beijing and Shenzhen in 2019

Chart 3: monthly average AQI comparison in Beijing from 2017 to 2019

3. Bottle web page application

3.1 folder structure

3.2 routin

3.3 template implementation

3.4 start the web service

4. Integrate Bokeh and Bottle

4.1 template modification

4.2 Python Code Integration

5. Deploy applications to Heroku

6. Reference documentation

In data science, it is a very important work to visualize data through charts. Before starting data analysis, data visualization can help us understand data, and more importantly, after completing the process of analysis, prediction and so on, we need to show the conclusions through data visualization. Creating interactive charts through web pages is an important means of displaying data.

1. Focus of the article and introduction of the project

The focus of this article will be to show how to integrate bokeh and bottle together and deploy them to the server for others to access, so we won't cover all the details of the implementation of bokeh and bottle, as well as the specific implementation details of pandas, but we will still explain the code we implement (probably not in that depth). This article will select the AQI data of China from 2017 to 2019 as the data set of the project, then use these data to draw three tables (a line chart and two bar charts with grouping), then build a presentation web page through bottle and bootstrap front-end template, and finally deploy the web application to Heroku (this step can be used as a reference, you can access local services through localhost, or select other cloud service providers' servers).

The dataset and code implementation used in this article can be found in the following github repository:

Https://github.com/pythonlibrary/bokeh-bottlepy

I have cleaned up the dataset, which contains regular daily AQI averages for each city from 2017 to 2019.

two。 Dataset research and chart preparation

In this section, like most data analysis projects, we will use jupyter notebook as our environment, because this tool can easily implement code changes and timely presentation of code results.

The most important thing is to import the necessary python libraries first.

Import numpy as npimport pandas as pdfrom bokeh.plotting import figure, showfrom bokeh.models import ColumnDataSource, HoverToolfrom bokeh.transform import dodgefrom bokeh.io import output_notebook

Then run in notebook, the following code to initialize bokeh,bokeh can output the icon to a different format file, such as html, etc., but in order to display in notebook, you need to specify at the beginning.

Output_notebook ()

After successful execution, notebook prompts you to load the bokeh environment successfully, as follows:

2.1 Import dataset

We use pandas's read_csv method to read into the data set and take out the data of some cities, because the data format of the date column after reading is not pandas datatime, so we make a conversion here to facilitate the use of later drawings, because there are some other air quality indicators in the data set, such as PM2.5, etc., we only select AQI as the focus to form a new data frame df.

Cities = ['Shanghai', 'Beijing', 'Hangzhou', 'Ningbo', 'Baoding', 'Nanjing', 'Suzhou', 'Shenzhen', 'Xiamen' Df = pd.read_csv ('AQI_merged.csv') df [' date'] = pd.to_datetime (df ['date']) df = df.sort_values (by='date'). Reset_index (drop=True) df = df [df [' type'] = = 'AQI']

Once our basic data frame (Dataframe) is created, let's take a look at what data it contains. We use the following code to extract the data from 2019 and show the first five records in notebook.

Df_2019_day = df [df ['date'] > =' 2019-01-01'] df_2019_day.head ()

As you can see, in the data frame, the AQI values of several cities are recorded according to the row of each day.

2.2 drawing a chart

Using the imported data, we will draw three charts:

Daily AQI variation curve of Shanghai, Beijing and Shenzhen in 2019 (curve)

Monthly average AQI comparison of Shanghai, Beijing and Shenzhen in 2019 (bar chart)

Monthly average AQI comparison in Beijing from 2017 to 2019 (bar chart)

Chart 1: daily AQI change curve of Shanghai, Beijing and Shenzhen in 2019

Using the df_2019_day data frame we just created, use the following code to draw the chart. Notice that we use the ColumnDataSource method provided by Bokeh to pass data to the bokeh chart.

To explain briefly: we first use figure to create an empty picture, then use the line method to draw the data of Shanghai, and then repeat the line method to add the data of the other two cities to the picture twice. Finally, we add a mouse hover prompt through the add_tools method to display the AQI value of the mouse position.

Source= ColumnDataSource (df_2019_day) p = figure (datetime), title= "average daily variation curve of AQI in 2019", plot_width=900, plot_height=400) p.line ('date',' Shanghai', line_color='blue', legend_label=' Shanghai', source=source) p.line ('date',' Beijing', line_color='green', legend_label=' Beijing', source=source) p.line ('date',' Shenzhen', line_color='orange') Legend_label=' Shenzhen', source=source) p.legend.location = "top_right" p.add_tools (HoverTool (tooltips= [("AQI", "$y")]) show (p)

Chart 2: comparison of monthly average AQI among Shanghai, Beijing and Shenzhen in 2019

We want to draw the monthly average AQI, and the data frame contains the daily AQI, so using dataframe's groupby method, we can get the monthly average. And a new column of month is created to store monthly information. Finally, through the head method to check whether the new data frame we obtained contains the monthly average AQI information.

Pd.options.mode.chained_assignment = Nonedf_2019_day ['month'] = df_2019_day [' date'] .apply (lambda x: x.strftime ('% Ymuri% m')) df_2019_month = df_2019_day.groupby (by='month'). Mean (). Reset_index () df_2019_month.head ()

The result of the dataset processing is in line with our expectations, and then use this dataset to draw a second chart. Because we want to compare the AQI of different cities in the same month, our bar chart needs to be displayed in groups, using the dodge mode in bokeh, where each dodge is the data of a city and indicates the relative position on the chart.

Source= ColumnDataSource (df_2019_month) p = figure (x_range=list (df_2019_month ['month']), title= "2019 AQI", plot_width=900, plot_height=400) p.vbar (x=dodge (' month',-0.25, range=p.x_range), top=' Shanghai', width=0.2, color= "# c9d9d3", legend_label= "Shanghai", source=source) p.vbar (x=dodge ('month', 0, range=p.x_range), top=' Beijing', width=0.2, color= "# 718dbf" Legend_label= "Beijing", source=source) p.vbar (x=dodge ('month', 0.25, range=p.x_range), top=' Shenzhen', width=0.2, color= "# e84d60", legend_label= "Shenzhen", source=source) p.xgrid.grid_line_color = Nonep.y_range.start = 0p.add_tools (tooltips= [("time", "@ month"), ("Shanghai average AQI", "@ {Shanghai}"), ("Beijing average AQI" "@ {Beijing}"), ("Shenzhen average AQI", "@ {Shenzhen}")]) show (p)

Chart 3: monthly average AQI comparison in Beijing from 2017 to 2019

Similar to figure 2, we do the necessary processing of the data frames, and because we want to show the comparison of different years and months, we put the year and month separately in the year and month columns.

Df ['date_ym'] = df [' date'] .apply (lambda x: x.strftime ('% YMUI% m')) df_month = df.groupby (by='date_ym'). Mean (). Reset_index () df_month ['month'] = df_month [' date_ym'] .apply (lambda x: x.split ('-') [- 1]) df_month ['year'] = df_month [' date_ym'] .apply (lambda X: x.split ('-') [0]) df_month.head ()

Then create three data frames, each containing only one year's data

Df_2017 = df_ month [DF _ month ['year'] = =' 2017'] [['month',' Beijing']] df_2018 = df_ month [DF _ month ['year'] = =' 2018'] [['month',' Beijing']] df_2019 = df_ month [DF _ month ['year'] = =' 20119] [['month',' Beijing']

Finally, a new bar chart is drawn using the same bokeh method.

Source_2017 = ColumnDataSource (df_2017) source_2018 = ColumnDataSource (df_2018) source_2019 = ColumnDataSource (df_2019) p = figure (x_range=list (df_2017 ['month']), title= "Beijing AQI comparison 2017-2019", plot_width=900, plot_height=400) p.vbar (x=dodge (' month',-0.25, range=p.x_range), top=' Beijing', width=0.2, color= "# c9d9d3", legend_label= "2017", source=source_2017) p.vbar (x=dodge ('month', 0) Range=p.x_range), top=' Beijing', width=0.2, color= "# 718dbf", legend_label= "2018", source=source_2018) p.vbar (x=dodge ('month', 2018, range=p.x_range), top=' Beijing', width=0.2, color= "# e84d60", legend_label= "2019", source=source_2019) p.xgrid.grid_line_color = Nonep.y_range.start = 0p.add_tools (tooltips= [("time", "@ month"), ("AQI") "@ {Beijing}")]) show (p)

So far, our three diagrams are ready, but they are all running in notebook. Later, we will simply transform the code and embed it into the bottle web application.

3. Bottle web page application

Bottle is an ultra-lightweight python web framework. The reason why we chose bottle instead of flask or Django in this article is that it is ultra-lightweight and can quickly build web applications. For web applications that only show data, using bottle allows you to get started quickly and focus on data analysis.

We will use bootstrap front-end template and bottle built-in template engine to implement this application. In order to achieve this goal quickly, we chose https://github.com/arsho/bottle-bootstrap as our initial code. Therefore, 99% of the web application code used in this project comes from this project, and we have only made a few changes. In this section, we will explain the key codes and concepts of bottle applications.

The corresponding code for this article can be found in the https://github.com/pythonlibrary/bokeh-bottlepy repository.

3.1 folder structure

The directory structure of our bokeh-bottlepy project is as follows, where

Dataset folder: contains the dataset csv file

Static folder: contains the bootstrap front-end framework code, including css,JavaScript, fonts, etc., used to display html pages with the theme of bootstrap

Views folder: contains the template of how we are going to display the data. As a starter project, this project contains only an index.tpl file, which is the template for the only single-page web page we have. The template will be rendered by importing data from the bottle application, which will always form the page seen by the user.

App.py: for our entry file, all our python code will be implemented in this, and finally run through: python app.py to start the service

Procfile: when it comes to Heroku deployment, we'll talk about it later

3.2 routin

Python web framework is used to realize dynamic web pages, that is to say, web pages are generated when users visit. The concept of routing is strange to people who come into contact with web applications for the first time, but it is actually very simple. Generally speaking, when a user clicks a link or button on a web page, or visits a link in the browser address bar, the web server will do different actions according to the link. The result is organized into html and presented to the user, this process is routing.

To implement routing in bottle is to implement a corresponding processing method for each url. The following code is all the relevant parts used in this project.

Dirname ='. 'app = Bottle () debug (True) @ app.route (' / static/') def send_css (filename): return static_file (filename, root=dirname+'/static/asset/css') @ app.route ('/ static/') def send_js (filename): return static_file (filename) Root=dirname+'/static/asset/js') @ app.route ('/') def index (): data = {"developer_organization": "pythonlibrary.net"} return template ('index', data = data)

All bottle web applications need to instantiate a Bottle object. As the service itself, we name it app and turn on debug mode, that is, when visiting url, the Bottle application will print some debugging information to help developers locate the problem.

The routing function is specified through the @ app.route decorator. The parameter of this decorator is relative url. For example, the routing address of the index function is /. If the local service port is 8080, the absolute url is: http://localhost:8080/. When the user accesses this address, the index function will be called, and its return value is the page the user sees. Here, the template method is used to render the template using data data. The concept of templates will be introduced in the next chapter.

To make a beautiful page, we need to use complex JavaScript and css. Fortunately, we choose the bootstrap framework to implement these complex parts for us. We only need to use the modules it provides to build a beautiful website.

In html, JavaScript and css are also accessed through url, so if you want the template to take effect, you need to tell bottle which path these JavaScript and css need to find locally. The send_css and send_js functions in the code use the static_file function in bottle to tell the application where the local resources are, and the routing address above is the address in the html when the user visits the web page, so these two functions are implemented. The connection between url and local resources.

3.3 template implementation

All Python web page frameworks, when developing web applications without the separation of front and rear, will contain the concept of a template. Most of these frameworks inherit their own template engine, and bottle also integrates a simple template engine they call SimpleTemplate. Of course, you can choose to use other third-party template engines, such as nijia2,mako.

In fact, the so-called template engine, even based on the replacement of template keywords, the engine provides a series of syntax, the engine can parse these syntax and make corresponding actions, such as filling in different data according to different circumstances, doing loops, judging, etc., and then the rest of the content will remain unchanged in the output, which can be compared through python's stringtemplate.

In our project, index.tpl is the template, which contains the syntax that SimpleTemplate can recognize and other contents, when SimpleTemplate parses the overall syntax of index.tpl, and fill in the appropriate data, you will eventually get the complete html content, so the template is the collection of html + engine syntax, as for the file suffix tpl is irrelevant, it can make any suffix you define, just general tpl represents template.

We made some changes to the file of the original code: modify the information in the head tag according to our project

China AQI

Then modify the navigation bar navbar div into our own link according to our requirements, and change the top text box in the main container of the page into our project description.

Home On Github China AQI data Visualization

This is a sample project based on a data visualization deployment of bottlepy, bokeh and Bootstrap, using China's AQI information data from 2017 to 2019 as the demonstration data of the project.

Back to app.py, the following code in this file implements the rendering of the index template through the template method. The parameter data of this method will be dynamically passed into the template as data. There is a {{data ["developer_organization"]}} statement in the corresponding template, which is the template syntax, which is similar to the python syntax. The value corresponding to the developer_ arrangement key in the data variable is accessed by dict.

@ app.route ('/') def index (): data = {"developer_organization": "pythonlibrary.net"} return template ('index', data = data) 3.4 start the web service

We have implemented an entry like the following in app.py. If you run python app.py in the terminal, this code will be executed and the web service can be started. The port of the service is 8080, and setting host to 0.0.0.0 means that other computers can access the services on this computer. If you only want local access on this computer, you can set it to localhost.

If _ _ name__ = "_ _ main__": port= 8080 app.run (host= "0.0.0.0", port=port, debug=True) 4. Integrate Bokeh and Bottle together 4.1 template modification

First of all, if we want to display the chart generated by bokeh in html, we need to load the JavaScript of bokeh and import it by adding the following CDN to index.tpl.

Then we will add the placeholder for the data chart (the associated engine syntax code), which will be dynamically replaced with the content provided in the python code when the template is rendered.

Add placeholders for three charts in the page body container

Note: unlike other data input syntax, there is an exclamation point (!) before data ["lot1_div"]. ), this is very important, if there is no exclamation mark, it means that the incoming data will be considered a string and will be enclosed in quotation marks when rendering, and what we actually want to populate here is the html code, not the html code enclosed in double quotation marks. The exclamation mark tells the engine that we pass in html or JavaScript or css code that the browser can handle.

Data ["plot1_div"]}} {{! data ["plot2_div"]}} {{! data ["plot3_div"]}}

Add the JavaScript script placeholder used to draw the chart after the body tag

{{! data ["plot_script"]}}

The future chart div and JavaScript scripts used in the template here will be generated by bokeh and rendered by bottle, which we will explain in the added section.

4.2 Python Code Integration

The code debugged successfully in notebook in chapter 2.2 will be converted into functions and implemented in app.py, note that we used the show (p) method to display the chart in notebook originally, in the web application we only return the chart object through return p, and the return value will be processed by the method provided by bottle.

Def get_df_from_source ():''get dataframes from the source dataset, only take the data of some big cities' cities = 'Shanghai', 'Beijing', 'Hangzhou', 'Ningbo', 'Baoding', 'Nanjing', 'Suzhou', 'Shenzhen', 'Xiamen' Df = pd.read_csv (dirname+'/dataset/AQI_merged.csv') df ['date'] = pd.to_datetime (df [' date']) df = df.sort_values (by='date'). Reset_index (drop=True) df = df [df ['type'] = =' AQI'] return dfdef draw_daily_AQI (mini_date) Df): year = mini_date.split ('-') [0] df_day = df [df ['date'] > = mini_date] source = ColumnDataSource (df_day) p = figure (xanthaxisymtype = "datetime", title= "{} annual AQI average variation curve" .format (year), plot_width=1150, plot_height=400) p.line (' date', 'Shanghai', line_color='blue', legend_label=' Shanghai' Source=source) p.line ('date',' Beijing', line_color='green', legend_label=' Beijing', source=source) p.line ('date',' Shenzhen', line_color='orange', legend_label=' Shenzhen', source=source) p.legend.location = "top_right" p.add_tools (HoverTool (tooltips= [("AQI", "$y")]) return p def draw_month_AQI (mini_date Df): year = mini_date.split ('-') [0] df_day = df [df ['date'] > = mini_date] df_day [' month'] = df_day ['date'] .apply (lambda x: x.strftime ('% Ymurt% m') df_month = df_day.groupby (by='month'). Mean (). Reset_index () source = ColumnDataSource (df_month) p = figure (x) _ range=list (df_month ['month']) Title= "AQI 2019", plot_width=1150, plot_height=400) p.vbar (x=dodge ('month',-0.25, range=p.x_range), top=' Shanghai', width=0.2, color= "# c9d9d3", legend_label= "Shanghai", source=source) p.vbar (x=dodge ('month', 0, range=p.x_range), top=' Beijing', width=0.2, color= "# 718dbf", legend_label= "Beijing", source=source) p.vbar (x=dodge ('month', 0.25) Range=p.x_range), top=' Shenzhen', width=0.2, color= "# e84d60", legend_label= "Shenzhen", source=source) p.xgrid.grid_line_color = None p.y_range.start = 0 p.add_tools (HoverTool [("time", "@ month"), ("Shanghai average AQI", "@ {Shanghai}"), ("Beijing average AQI", "@ {Beijing}"), ("Shenzhen average AQI" "@ {Shenzhen}")) return pdef draw_year_AQI (df): df ['date_ym'] = df [' date'] .apply (lambda x: x.strftime ('% Ymure% m')) df_month = df.groupby (by='date_ym'). Mean (). Reset_index () df_month ['month'] = df_month [' date_ym'] .apply (lambda x: X) .split ('-') [- 1]) df_month ['year'] = df_month [' date_ym'] .apply (lambda x: x.split ('-') [0]) df_2017 = df_ month [DF _ month ['year'] =' 2017'] [['month'] 'Beijing']] df_2018 = df_ month [DF _ month ['year'] =' 2018'] [['month',' Beijing']] df_2019 = df_ month [DF _ month ['year'] = =' 2019'] [['month'] Source_2017 = ColumnDataSource (df_2017) source_2018 = ColumnDataSource (df_2018) source_2019 = ColumnDataSource (df_2019) p = figure (x_range=list (df_2017 ['month']), title= "Beijing AQI comparison 2017-2019", plot_width=1150, plot_height=400) p.vbar (x=dodge (' month',-0.25, range=p.x_range), top=' Beijing', width=0.2, color= "# c9d9d3", legend_label= "2017" Source=source_2017) p.vbar (x=dodge ('month', 0, range=p.x_range), top=' Beijing', width=0.2, color= "# 718dbf", legend_label= "2018", source=source_2018) p.vbar (x=dodge ('month', 2018, range=p.x_range), top=' Beijing', width=0.2, color= "# e84d60", legend_label= "2019" Source=source_2019) p.xgrid.grid_line_color = None p.y_range.start = 0 p.add_tools (HoverTool (tooltips= [("time", "@ month"), ("AQI", "@ {Beijing}")])) return p

The three drawing functions return the chart object p. What if we can let bottle render the chart object so as to display the chart on the web page? Bokeh provides a components method that takes the chart object as a parameter and returns the JavaScript script and chart div used for the drawing, so modify our index routing function to:

App.route ('/') def index (): df= get_df_from_source () plot1 = draw_daily_AQI ('2019-01-01, df=df) plot2 = draw_month_AQI (' 2019-01-01, df=df) plot3 = draw_year_AQI (df=df) plots_data = components ((plot1, plot2, plot3) data = {"plot_script": plots_data [0] "plot1_div": plots_data [1] [0], "plot2_div": plots_data [1] [1], "plot3_div": plots_data [1] [2], "developer_organization": "pythonlibrary.net"} return template ('index', data = data)

Here, the plot1_div,plot2_div,plot3_div and plot_script in the data dictionary in the index.tpl template will be replaced by dynamic rendering. Finally, it achieves the goal of displaying the chart on the web page.

You can try to run it by clone the warehouse of the project, or visit http://china-aqi-data-visulazition.herokuapp.com/ directly to see the effect.

5. Deploy applications to Heroku

This part of the content is not directly related to how to display the data chart on the web page, but is just an optional free cloud service that allows you to share your page or learn about web deployment. But different services may be deployed in different ways, so if you want to deploy your web page to another service provider, the knowledge here may not be applicable at all.

Users can deploy a limited number of web applications on Heroku for free, and the process is very simple. Just implement a Procfile file and the Heroku system will know how to run your service. Procfile in our project uses the following code, which is similar to our local running service.

Web: python app.py

For the entry code of app.py, you need to change port to read from environment variables, because Heroku will dynamically assign ports to the application. If you specify a fixed value, users will not be able to access the service because Heroku does not open its external access.

If _ _ name__ = "_ _ main__": port=int (os.environ.get ("PORT", 8080)) app.run (host= "0.0.0.0", port=port, debug=True

Finally, the user can choose to connect the github repository to the application on the Heroku page, and the system will automatically pull the latest code from github and start the service.

This is the end of the content of "how to use python data visualization Bokeh and Bottle.py to display your data on the web page". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.