Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to use pandas or numpy in python to deal with the null value of data, with a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let Xiaobian take you to understand.

My goal was to get all the non-zero and non-null values in the data, and then calculate the mean of all the data obtained, and then fill in the zero and null values with the mean. This operation made me aware of the difference between I is None/np.isnan (I) / i.isnull (), and here's a brief introduction:

1. About np.nan:

Let's make it clear that there is only one way to generate null values, namely, np.nan ().

Some wonderful properties of # np.nan (): np.nan = = np.nan > > False np.isnan (np.nan) > > True np.nan is None > False type (np.nan) > float summarize:

Np.nan is not an "empty" object, it is judged to be False by i is None.

To judge whether a value is null or not, you can only use the np.isnan (I) function, and never do it with I = = np.nan (), otherwise you will die miserably, because the null value cannot be correctly identified by the equal "=" (the first two items in the above example).

Np.nan is a non-empty object whose type is the basic data type float (isn't it amazing, I don't know why it's designed this way)

When to use 2.np.isnan () and pd.isnull (): # first create a DataFrame:bb = pd.DataFrame ({'aqiang: [0pjnp.nan]}) bb > a 00.0 11.02 2.0 3 NaN # first test np.isnan () np.isnan (bb) > a 0 False 1 False 2 False 3 True#. It is worth mentioning that if you want to know whether the whole DataFrame is empty or not. You can do this on this basis: np.isnan (bb). All () > > a False dtype: bool # this line refers to the returned value of dtype # and test isnull () pd.isnull (bb) > a 0 False 1 False 2 False 3 True

As you can see from the above, both np.isnan () and pd.isnull () can detect null values for either DataFrame, Python list, or just one value. But generally in practical applications, np.isnan () is mostly used for the test of a single value, and pd.isnull () is used for the test of a DataFrame or Series (whole).

In addition, according to the pandas official documentation and source code, another function provided by pandas, pd.isna (), is exactly the same as pd.isnull ().

For the any () / all () function mentioned above, see the pandas documentation:

Https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.all.html#pandas.DataFrame.all

Https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.any.html#pandas.DataFrame.any

Other functions related to null detection or deletion include notna (), fillna (), dropna (), and so on. It should be used flexibly in actual combat.

Add: nan in numpy (judging that an element is equal to nan, and nan security function)

NaN in Nunpy creates nan (null) import numpy as npnp.nannannp.NaNnannp.NANnan in multiple ways to determine whether there is a null value x = np.array ([1,1,8, np.nan, 10]) np.nan in xFalsenp.isin (np.nan, x) array (False) 1 in xTruenp.isin (1, x) array (True) np.isnan (x) array ([False, True, False]) x [- 2] = np.nan, np.isnan (x [- 2]) (False, True)

In np, nan needs to be identified by the function isnan, and note:

Xarray ([1, 1, 8, nan, 10.])

All elements in x become floating-point because nan is floating-point.

Nan security function np.mean (x) nannp.nanmean (x) 5.0

In addition, max, min, median, etc., are all non-nan secure by default, and nan is required to mark nan security.

Nan security is the default in ps:pandas.

Add: the way Python handles DataFrame data pd.isnull () np.isnan ()

When dealing with data, we often encounter null values in dealing with data, involving several common functions, such as pd.isnull (), pd.notnull (), np.isnan (), pd.notna (), pd.isna (), pd.fillna (), pd.dropna () and so on.

This article focuses on pd.isnull (), pd.notnull (), np.isnan (), pd.notna (), pd.isna ().

Summary:

As you can see below, both np.isnan () and pd.isnull () can detect null values for either DataFrame, Python list, or just one value. But generally in practical applications, np.isnan () is mostly used for the test of a single value, and pd.isnull () is used for the test of a DataFrame or Series (whole).

1.pd.isnull ()

Pd.isnull () can detect null values for either DataFrame, Python list, or just one value. But generally in practical applications, pd.isnull () is used to test a DataFrame or Series (whole).

In addition, according to the pandas official documentation and source code, another function provided by pandas, pd.isna (), is exactly the same as pd.isnull ().

# first create a DataFrame:df = pd.DataFrame ({'aqu: [0min1pyr2) Np.nan]}) df > > a 0 0.0 1 1.0 2 2.0 3 NaN # Test isnull () pd.isnull (df) > > a 0 False 1 False 2 False 3 True # Test isna () pd.isna (df) > > a 0 False 1 False 2 False 3 True # Test notnull () pd.notnull (df) > > a 0 True 1 True 2 True 3 False # Test Notna () pd.notna (df) > > a 0 True 1 True 2 True 3 False2.np.nan ()

Determine whether it is np.nan ().

Np.nan is not an "empty" object, and you can only use the np.isnan (I) function to determine whether a value is null.

Np.nan is a non-empty object whose type is the base data type float.

Np.nan () can detect null values for either DataFrame, Python list, or just one value. But generally in practical applications, np.nan () is often used to test a single value.

Np.nan = = np.nan > > False np.isnan (np.nan) > True type (np.nan) > > float np.nan is None > False np.isnan (df) > > a 0 False 1 False 2 False 3 True thank you for reading this article carefully. I hope the article "how to use pandas or numpy to deal with null data in python" shared by the editor will be helpful to you. At the same time, I hope you will support it and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.