Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you what are the skills for efficient data analysis of 8 Python. The article is rich in content and analyzed and described from a professional point of view. I hope you can get something after reading this article.

Whether you are participating in a Kaggle competition or developing a deep learning application, the steps are always data analysis.

This paper introduces eight methods of data analysis using Python, which can not only improve the running efficiency, but also make the code more "beautiful".

One line of code defines List

Writing a For loop is too cumbersome when defining a list, and fortunately, Python has a built-in way to solve this problem in a single line of code.

The following is a comparison between creating a list using a For loop and creating a list with a single line of code.

X = [1Jing 2 item**2 3 4] out = [] for item in x: out.append (item**2) print (out) [1, 4, 9, 16] # vs. X = [1 item**2 for item in x] print (out) [1, 4, 9, 16]

Lambda expression

Tired of defining functions that can be used only a few times? Lambda expression is your savior!

Lambda expressions are used to create small, one-time and anonymous function objects in Python, which can create a function for you.

The basic syntax of an lambda expression is:

Lambda arguments: expression

Be careful! As long as you have a lambda expression, you can do whatever a regular function can do.

You can feel the power of lambda expressions from the following example:

Double = lambda x: X * 2 print (double (5)) 10

Map and Filter

Once you have mastered lambda expressions, you can achieve more powerful functionality by learning to use them with Map and Filter functions.

Specifically, map performs some action on each element in the list and converts it into a new list.

In this case, it iterates through each element and multiplies it by 2 to form a new list. (attention! The list () function simply converts the output to a list type)

# Map seq = [1,2,3,4,5] result = list (map (lambda var: var*2, seq)) print (result) [2,4,6,8,10]

The Filter function accepts a list and a rule, just like map, but returns a subset of the original list by comparing each element with a Boolean filter rule.

# Filter seq = [1,2,3,4,5] result = list (filter (lambda x: X > 2, seq)) print (result) [3,4,5]

Arange and Linspace

Arange returns an isometric list of the given step size.

Its three parameters start, stop and step represent the start value, the end value and the step size, please note! The stop point is a "cut-off" value, so it is not included in the array output.

# np.arange (start, stop, step) np.arange (3,7,2) array ([3,5])

Linspace and Arrange are very similar, but slightly different.

Linspace divides the interval evenly by a specified number, so a given interval start and end, as well as the number of equal partition points num,linspace will return an NumPy array.

This is especially useful for data visualization and declaration of axes when drawing.

# np.linspace (start, stop, num) np.linspace (2.0,3.0, num=5) array ([2.0,2.25,2.5,2.75,3.0]

What does Axis stand for?

In Pandas, you may encounter Axis when deleting a column or summing values in a NumPy matrix.

Let's use an example of deleting a column (row):

Df.drop ('Column Aids, axis=1) df.drop (' Row Aids, axis=0)

If you want to process columns, set Axis to 1, and if you want to process rows, set it to 0.

But why?

Think back to shape in Pandas.

Df.shape (# of Rows, # of Columns)

Calling the shape property from Pandas DataFrame returns a tuple with * values representing the number of rows and the second value representing the number of columns.

If you want to index it in Python, the number of rows is subscript 0 and the number of columns is subscript 1, much like how we declare axis values.

Concat,Merge and Join

If you are familiar with SQL, these concepts may be easier for you.

In any case, these functions are essentially the way DataFrame is combined in a particular way.

It can be difficult to track which is the most suitable for use at which time, so let's review.

Concat allows users to append one or more DataFrame below or next to the table (depending on how you define the axis).

Merge merges multiple DataFrame into rows that specify the same primary key (Key).

Join, like Merge, merges two DataFrame.

However, it does not merge by a specified primary key, but by the same column or row name.

Pandas Apply

Apply is designed for Pandas Series.

If you are not familiar with Series, think of it as an array similar to Numpy.

Apply applies a function to each element on the specified axis.

With Apply, you can format and manipulate the value of a DataFrame column (which is a Series) without looping, which is very useful!

Df = pd.DataFrame ([[4,9],] * 3, columns= ['Aids,' B']) df A B 0 4 9 1 4 9 2 49 df.apply (np.sqrt) A B 0 2.0 3.0 1 2.0 2 2.0 3.0 df.apply (np.sum, axis=0) A 12 B 27 df.apply (np.sum, axis=1) 0 13 1 13 2 13

Pivot Tables

If you are familiar with Microsoft Excel, you may have heard of PivotTable reports.

Pandas's built-in pivot_table function creates a spreadsheet-style PivotTable in the form of DataFrame, which helps us quickly view certain columns of data.

Here are a few examples:

Very intelligently group the data according to "Manager":

Pd.pivot_table (df, index= ["Manager", "Rep"])

Or you can filter attribute values.

Pd.pivot_table (df,index= ["Manager", "Rep"], values= ["Price"])

I hope the above description will help you find some useful functions and concepts in Python.

These are the eight Python efficient data analysis techniques shared by the editor. If you happen to have similar doubts, please refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.