How to use several lines of code to make feature selection
From sklearn.feature_selection import RFEfrom sklearn.linear_model import LinearRegression#Load boston housing dataset as an exampleX = np.array (train1 1 [feature _ use]. Fillna (- 1)) [1Partition:] Y = np.array (train1 ['target']) [1:train1.size] # print (X) # print (Y) names = feature_use#use linear regression as the modellr = LinearRegression () # rank all features, i.e continue the elimination until the last onerfe = RFE (lr, n_features_to_select=1) rfe.fit (X) Y) print ("Features sorted by their score:") # print (sorted (map (lambda x: round (x, 4), rf.feature_importances_), names), reverse=True)) sortedlist = sorted (map (lambda x: round (x, 4), rfe.ranking_), names) Reverse=True) print (sortedlist) feature_use = [] for index in sortedlist [len (sortedlist)-70:]: if index [0] > 0: feature_use.append (index [1]) print (feature_use)
The above X is the feature set of the dataset, Y is the label set.
Ranking the importance of features in sortlist
Recently, one of the insights of machine learning is that the influence of features is much greater than that of model parameters, and the feature is the reflection of the real world in the algorithm.
In feature engineering, we should have a very deep understanding of the business, emphasize returning to nature, delete invalid features, and reduce the features that cause interference.
The process of adding features requires one by one, and we need to think more about whether the relationship between these features is strongly linear.
# random forest select features'''from sklearn.ensemble import RandomForestRegressorimport numpy as np#Load boston housing dataset as an exampleX = np.array (room1 [feature _ use] .fillna (- 1)) [1Participation 1.sizeMagna:] Y = np.array (train1 ['target']) [1:train1.size] print (X) print (Y) names = feature_userf = RandomForestRegressor () rf.fit (X, Y) print ("Features sorted by their score:") print (sorted (zip (map (lambda x: round (x, 4)) Rf.feature_importances_), names), reverse=True)''
