Shulou(Shulou.com)06/02 Report--

This article focuses on "how to use Python to do data filtering", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "how to use Python for data filtering".

EXCEL screening

Let's review the data requirements in the previous article. The first is to screen the 2010 orders according to the order date, and the second is to filter the high-level orders according to the order level. This time, we upgrade the filter criteria and add one condition: on the basis of the previous two screening criteria, we select the top 10 orders in the profit amount. If you operate in EXCEL, you can use the formula or its own filtering function to achieve it, but how is it implemented in python?

We can write python scripts on pycharm and other software to achieve, but I recommend that you use the ETL function of Smartbi intelligence analysis to achieve, Smartbi intelligence analysis ETL interface also provides python script writing page, and many simple requirements can be realized through ETL, no need to write complex code, very suitable for programming rookies.

Load data

Open the ETL interface in Smartbi Intelligence Analysis. We first connect the data source. I have imported the EXCEL file into SmartBI Intelligence Analysis system ahead of time, then drag the relational data source to the display area, and find the path where the data source is stored on the right:

After loading, we can preview the data source:

Conditional filtering

After the data connection is completed, you can use the python script included in Smartbi Intelligence Analysis ETL to process the data. We first drag the components of python to the middle display area and connect them to the above relational data source:

The following script is included with Smartbi Intelligence Analysis. The libraries of pandas and numpy have been installed in advance, and part of the code has been written for the following function. As long as you extend it according to the following script, you can use python to clean the data on ETL:

Before entering the python script, let's be clear about the requirements to be implemented, the first two conditions are the same as last time, we do not need to change, just copy the script, if you do not understand, you can see my previous article. The third condition is to filter the top 10 profit orders. Here we use the sort_values function in pandas to sort. The principle is similar to order by in SQL. The dataset can be sorted according to the data in a certain field, followed by ascending=False, that is, the data is sorted in descending order. Head (10) represents the value of the top 10:

Let's take a look at the effect of the execution of the python script. The year column filters out 2010, the order column filters advanced orders, and the order profit column filters out the top 10 data in descending order:

Data output

After completing the filtering procedure of python, we save the filtered table, either in the database or in the dataset. Here, I still choose the saving method of the dataset, because it is convenient to call in EXCEL:

When you go back to EXCEL, click "View" in the EXCEL plug-in, and the dataset panel interface of Smartbi Intelligence Analysis will appear on the right. Click the drop-down menu to find the dataset saved in the cloud, or you can enter the file name to search:

Click "Refresh" on the EXCEL toolbar, and the fields in the dataset will be displayed in EXCEL, and we can use these data for data analysis in EXCEL:

At this point, I believe you have a deeper understanding of "how to use Python to do data filtering". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.