Shulou(Shulou.com)05/31 Report--

This article will explain in detail the example analysis of Python Zhongling regression. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Ridge regression is a biased estimation regression method specially used for collinear data analysis. In essence, the improved least square estimation method, by giving up the unbiasedness of the least square method (in the case of repeated sampling, the expectation of the sample mean set is equal to the population mean), a more realistic and reliable regression method can be obtained at the cost of losing part of the information and reducing the accuracy. The fitting of collinear problems and ill-posed data is better than the least square method, and it is often used in multi-dimensional problems and ill-posed problems (ill-posed problem).

Ridge regression solves the problem of ordinary least square method by introducing a penalty variable. The correlation coefficient of ridge regression is the minimum sum of squares of penalty residuals:

It is the shrinkage, a factor that controls the complexity of the model.

Import numpy as npimport matplotlib.pyplot as pltfrom sklearn import linear_model# creates a Hilbert matrix (highly morbid, where the points of any element change The addition operation here is similar to the matrix multiplication X = 1. / (np.arange (1,11) + np.arange (0,10) [:, np.newaxis]) y = np.ones (10) # calculate the path n_alphas = 200alphas = np.logspace (- 10,-2) N_alphas) clf = linear_model.Ridge (fit_intercept=False) coefs = [] for an in alphas: clf.set_params (alpha=a) clf.fit (X, y) coefs.append (clf.coef_) # graphic display # set scale ax = plt.gca () # set scale mapping ax.plot (alphas Coefs) # set the scale display of the x-axis ax.set_xscale ('log') # flip the x-axis ax.set_xlim (ax.get_xlim () [:-1]) # set the x and y labels and the title plt.xlabel (' alpha') plt.ylabel ('weights') plt.title (' Ridge coefficients as a function of the regularization') # so that the maximum and minimum values of the axis are consistent with the data plt.axis ('tight') plt.show ()

The above figure shows the trend that the 10 components of the solution of the ridge regression model vary with the regularization parameter Alpha. Each color represents a different correlation coefficient vector feature, which varies with the change of the incoming regularization parameter Alpha. Because of the image shape, ridge regression is also called ridge regression.

This example shows the advantage of ridge regression in dealing with ill-conditioned matrix (ill-conditioned matrices). A small change in each target variable in the ill-conditioned matrix will produce a huge variance. In this case, it is necessary to set a more appropriate regularization parameter to reduce the deviation (noise).

When the regularization parameter Alpha is very large, the influence of regularization dominates the square function, and the correlation coefficient approaches to 0. At the end of the path, when the regular parameter alpha approaches to 0, the resulting solution approaches to the ordinary least square method, and the coefficient shows a great oscillation.

Set regular parameters

In practice, it is necessary to constantly adjust the regular parameter Alpha to find a balance in the above process.

RidgeCV implements cross-validation of ridge regression. The following is an efficient cross-validation method-leave a cross-validation (leave-one-out):

> from sklearn import linear_model > clf = linear_model.RidgeCV (alphas= [0.1,1.0,10.0]) > clf.fit ([[0,0], [0,0], [1,1]], [0,1.0,1]) RidgeCV (alphas= [0.1,1.0,10.0], cv=None, fit_intercept=True, scoring=None Normalize=False) > clf.alpha_ 0.1The article "sample Analysis of Ridge regression in Python" ends here. Hope that the above content can be helpful to you, so that you can learn more knowledge, if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.