Shulou(Shulou.com)06/02 Report--

This article is to share with you about how to use missingno in Python. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

Install and reference pip install missingno

Import missingno as msno

Missingno is generally used with numpy and pandas:

Import numpy as np

Import pandas as pd

Quilt is also used in the case, which is a packet manager that allows you to manage data like management code. Anaconda is not built-in, so it needs to be installed.

Import data from quilt.data.ResidentMario import missingno_data

Collisions = missingno_data.nyc_collision_factors ()

Collisions = collisions.replace ("nan", np.nan)

# # drawing matrix diagram of missing values

Msno.matrix invalid matrix is a data-intensive display, which can quickly and intuitively see the integrity of the data.

Msno.matrix (collisions.sample)

The more blanks, the more serious the missing.

The mini-chart on the right outlines the general shape of data integrity and indicates the number of rows in the dataset with maximum and minimum invalid values. It is important to note that this matrix holds up to 50 variables, and tags beyond this range begin to overlap or become unreadable, and large displays ignore them by default.

If you are working with time series data, you can use keyword parameters to specify periodic freq:

Null_pattern = (np.random.random (1000) .reshape ((50,20)) > 0.5) .astype (bool)

Null_pattern = pd.DataFrame (null_pattern) .replace ({False: None})

Msno.matrix (null_pattern.set_index (pd.period_range ('1gamma 2011, freq='M')), freq='BQ')

! [] (https://my-wechat.oss-cn-beijing.aliyuncs.com/image_20200403162803.png)

Draw a bar chart of missing values

Bar charts provide the same information as matrix charts, but in a simpler format. Msno.bar (collisions.sample (1000))

Draw the missing value thermal map

Missingno correlation thermal maps can show invalid correlations: the existence of one variable or the absence of how strongly affecting the existence of another.

The value is 1: two variables, one is missing and the other must be missing.

The value is-1: one variable is missing and the other must not be missing.

The value is 0: the presence or absence of missing values of variables does not affect each other.

The thermal map is very suitable for selecting the data integrity relationship between variable pairs, but when it comes to larger relationships, its explanatory power is limited, and it does not particularly support very large data sets.

Note: variables that are always full or empty do not have any meaningful association and are therefore removed from the visualization.

Msno.heatmap (collisions)

Missing value tree

Through the tree view, we can observe the correlation of the missing variables more comprehensively and reveal a deeper correlation than the correlation thermal map.

Msno.dendrogram (collisions)

The above is how to use missingno in Python. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.