Shulou(Shulou.com)06/01 Report--

This article analyzes "how to analyze sklearn basics and data processing". The content is detailed and easy to understand. Friends interested in "how to analyze sklearn foundation and data processing" can read it slowly and deeply along with the ideas of Xiaobian. I hope it can help everyone after reading. Let's learn more about "how to analyze sklearn basics and data processing" together with Xiaobian.

Sklearn library integrates multiple machine learning algorithms to quickly build models during data analysis. Although the Pandas library already provides data merging, cleaning, and normalization (dispersion normalization, standard deviation normalization, and fractional scaling normalization), more preprocessing operations are needed to build machine learning models for data features, so sklearn encapsulates the relevant preprocessing functions into a unified interface--Transformer. The sklearn converter enables normalization, binarization, PCA will be, etc. of the incoming NumPy array.

When it comes to transforming data, the padas library actually provides functions such as processing categorical data with dummy variables and discretizing continuous data. This is one of the reasons why learning SQL alone cannot completely replace the Pandas feature. However, sklearn introduces converters to make it easier to unify the training set and test set operations.

Sklearn also provides classic datasets that are easy to learn, and these datasets are stored like dictionaries. These data and their values can be visually seen through the Spyder variables interface in ancanda. From this data we can also understand what the data format ultimately looks like before data analysis. For example: data (data), label (target), feature (feature) and other three basic elements. Subsequent training sets and tests are split and trained without these preparatory data.

1. Load datasets

If you need to load a dataset, you can assign the corresponding function to a variable, emphasizing again the three elements of the dataset: data, target, and feature. As shown in the following code:

from sklearn.datasets import load_breast_cancer

cancer = load_breast_cancer()##Assign dataset to iris variable

print ('breast_cancer length of dataset is:', len(cancer))

print ('breast_cancer DataSet type is:', type(cancer))

cancer_data = cancer['data']

print ('breast_cancer dataset data is: ','\n', cancer_data)

cancer_target = cancer ['target '] ##Remove label from dataset

print ('breast_cancer dataset with label: \n', cancer_target)

cancer_names = cancer ['feature_names']##Take the feature names of the dataset

print ('breast_cancer dataset feature names are: \n', cancer_names)

cancer_desc = cancer ['DESCR '] ##Get the description of the dataset

print ('breast_cancer dataset description information is: \n', cancer_desc)

Divide data into training set and test set

Why is the data split? Because that's where the machine learning approach comes in. Let computational thinking discover relationships within data. This method is unlike traditional experimental and theoretical thinking. The machine learning idea is to find out the inherent laws and relationships of data according to a given label training set.

Sklearn's model_selection module provides the train_test_split function, which splits the dataset.

print ('The original dataset data has the shape:', cancer_data.shape)

print ('Original dataset label has the shape:', cancer_target.shape)

from sklearn.model_selection import train_test_split

cancer_data_train, cancer_data_test,\

cancer_target_train, cancer_target_test = \

train_test_split(cancer_data, cancer_target,

test_size=0.2, random_state=42)

print ('The shape of the training set data is:', cancer_data_train.shape)

print ('training set label shape is:', cancer_target_train.shape)

print ('The shape of the test set data is:', cancer_data_test.shape)

print ('The shape of the test set label is:', cancer_target_test.shape)

3. Data preprocessing and dimensionality reduction through sklearn converter

In order to eliminate the possible effects of dimensional and range differences between features, data normalization is required, also known as normalization. In fact, normalization is the process of reducing spatial complexity, and PCA dimensionality reduction corresponds to the process of reducing temporal complexity.

Sklearn's converter consists of three main methods: fit, transform, fit_transform, etc.

Zhengzhou infertility hospital: http://www.xbzztj.com/

import numpy as np

from sklearn.preprocessing import MinMaxScaler

Scaler = MinMaxScaler().fit(cancer_data_train) ##Generate rules

##Apply rules to training set

cancer_trainScaler = Scaler.transform(cancer_data_train)

##Apply rules to test sets

##cancer_testScaler = Scaler.transform(cancer_data_test)

Scaler = MinMaxScaler().fit(cancer_data_test) ##Generate rules

cancer_testScaler = Scaler.transform(cancer_data_test)

print ('Minimum value of training set data before dispersion normalization is:', np.min(cancer_data_train))

print ('Minimum value of training set data after dispersion normalization is:', np.min(cancer_trainScaler))

print ('Maximum value of training set data before dispersion normalization is:', np.max(cancer_data_train))

print ('Maximum value of training set data after dispersion normalization is:', np.max(cancer_trainScaler))

print ('Minimum value of test set data before dispersion normalization is:', np.min(cancer_data_test))

print ('Minimum value of test set data after dispersion normalization is:', np.min(cancer_testScaler))

print ('Maximum value of test set data before dispersion normalization is:', np.max(cancer_data_test))

print ('Maximum value of test set data after dispersion normalization is:', np.max(cancer_testScaler))

from sklearn.decomposition import PCA

pca_model = PCA(n_components=10).fit(cancer_trainScaler) ##Generate rules

cancer_trainPca = pca_model.transform(cancer_trainScaler) ##Apply rules to training set

cancer_testPca = pca_model.transform(cancer_testScaler) ##Apply rules to test sets

print ('The shape of the training set data before PCA dimensionality reduction is:', cancer_trainScaler.shape)

print ('The shape of the training set data after PCA dimensionality reduction is:', cancer_trainPca.shape)

print ('The shape of the test set data before PCA dimensionality reduction is:', cancer_testScaler.shape)

print ('PCA dimensionality reduction test set data shape is:', cancer_testPca.shape)

About how to analyze sklearn foundation and data processing to share here, I hope the above content can let everyone improve. If you want to learn more, please pay more attention to the updates of Xiaobian. Thank you for your attention to the website!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.