Shulou(Shulou.com)06/01 Report--

Today, the editor will share with you the relevant knowledge points about how to achieve the normalization algorithm in Python. The content is detailed and the logic is clear. I believe most people still know too much about this, so share this article for your reference. I hope you can get something after reading this article. Let's take a look at it.

1. Preface

The normalization algorithm Normalization processes the data into dimensional data all the time, which is generally limited to [0mem1] and [- 1mem1].

Generally speaking, data normalization is needed in modeling.

The reasons are as follows:

Reduce the difficulty of calculation

It is possible to improve the prediction accuracy of the model.

Eliminate dimensional influence

Here are three common standardization methods, namely, maximum and minimum, normal centralization, and decimal point calibration

2.Min-Max method 2.1Formula

2.2 algorithm implementation logic

1. Find the maximum and minimum values of a set of data

two。 Using Formula normalization

3. Enter the result (to facilitate visual display, we have designed the code)

2.3Code import numpy as npimport pandas as pdimport matplotlib.pyplot as plt# 1. Min-Max Normalized Data = np.array ([0.2 max 0.9 data 29], [0.9 mai 0.1100], [0.5 min 0.100]]) # Minimax normalization algorithm # 1.1 data conversion def MinMax (data): Min = 0 max = 1 C = data [: 2] min = np.min (C) max = np.max (C) for one in data: one [2] = (one [2]-min) / (max-min) print ('transformed matrix:\ nChart data) return data# 1.2 Visual def ShowData (Data) ShowD1): length = len (Data) X = np.ones (Data.shape [0]) plt.figure (1) plt.subplot (121) for i in range (length): plt.scatter (X* (iTun1), Data [:, I]) plt.subplot (122) for i in range (length): plt.scatter (X* (iTun1), ShowD1 [:, I]) plt.show () ShowData (Data,MinMax (Data.copy ()

Transformed matrix:

[[0.2 0.9 0. ]

[0.9 0.1 1.]

[0.5 0.5 0.01408451]]

2.4 limitations

The data scale is too large to adapt.

After normalization, the range of data is [0quoi 1], so we should use it cautiously for some raw data with positive numbers.

3 Z-score standardized 3.1Formula

3.2 algorithm implementation logic

Input data

The mean value and method of the data are calculated by using the centralization formula.

Output result

Code def Zscore (data): x_mean = np.mean (data [: 2]) length = len (data [:, 2]) vari = np.sqrt ((np.sum ((data [: 2]-x_mean) * * 2)) / length) print ('Variance:', vari) data [:, 2] = (data [:, 2]-x_mean) / vari print ('Z-score standardized matrix is' Data) return dataShowData (Data,Zscore (Data.copy ()

Variance: 51.569160680908254

The standardized matrix of Z-score is [0.20.9 0.13864876]

[0.9 0.1 1.5154406]

[0.5 0.5 0.15804019]]

3.4 limitations

Poor performance for data with a small sample size

After standardization, the range is positive, and the range is in [- 1pd1]

4Formula 4.1 of decimal calibration

4.2 algorithm implementation logic

Input data

Absolute value, maximum value

Divide each number by the order of magnitude of the maximum

Input data

Code implementation # decimal scaling normalization algorithm def Decimals (data): C = np.abs (data [:, 2]) max = int (np.sort (C) [- 1]) # sort from small to large, take the last place And the maximum value k = len (str (max)) print ('the number of digits of the maximum absolute value:\ nDefinery k) data [: 2] = data [:, 2] / (10 decimal point) print (' Matrix after normalization of the decimal point:\ nThe normalized matrix (Data,Decimals (Data.copy ()

The number of digits with the highest absolute value:

three

The matrix after the normalization of the decimal point:

[[2.9e-02 1.0e-01 3.0e-02]

[2.9e-02 1.0e-01 3.0e-02]

[5.0e-01 5.0e-01 3.0e+01]]

4.4 limitations

Be greatly affected by the maximum value.

These are all the contents of the article "how to achieve the normalization algorithm in Python". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.