Shulou(Shulou.com)06/01 Report--

This article will explain in detail about the regularization of the loss function in big data. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Regularization of loss function

Norm: used to measure the length or size of each vector in a vector space (or matrix).

Vector norm-L1 and L2 are the most commonly used

Matrix norm

Regularization of linear regression

When using the training set to fit the linear regression equation, such as using the polynomial fitting, generally speaking, the higher the degree of the fitting equation, the better the fitting effect, the smaller the loss function, but the fitting function also becomes more complex.

In some abnormal cases, such as a small amount of training data or too many features, if there are samples in the test set that are not available in the previous training set, the prediction results may not be effective, resulting in a large variance of the classification results and over-fitting, thus it can be seen that the loss function is too small to be desirable, which means that the generalization ability of the model needs to be improved, so regularization terms are added. The identification of the regularization parameter here is α, and it will be shown as λ in some data, because in the sklearn library, this parameter is identified as α, and for ease of use, it is identified as α.

Gradient updating method after regularization of linear regression

Same as the gradient renewal formula of linear regression, when θ k is equal to θ j, the derivative of the formula can be obtained and a new gradient renewal formula can be obtained.

Why use regularization?

Why

Why use regularization for regression?

In order to strengthen the understanding of regularization, an example is given to illustrate why regression should be regularized-Regularization.

The increase of the degree of polynomial has little effect on the performance of fitting.

When the number of training samples is reduced from 500 to 498, it is found that the effect of high-order polynomials on fitting decreases, the ability to predict data decreases, and the phenomenon of over-fitting occurs.

The basic method of regularization is to add the sum of the absolute values of the coefficients of all polynomials-L1 regularization, or the square sum of the absolute values of polynomials-L2 regularization to the penalty term, and formulate a penalty intensity factor to avoid abnormal coefficients, that is, by using Lasso regression-L1 regularization, ridge regression-L2 regularization, or elastic net regression-L1+L2 regularization to reduce overfitting. By using ridge regression as an example, we can find that the model overcomes the over-fitting problem caused by 100-degree polynomials.

The role of regular weight in Ridge regression

The regularization intensity α is the regularization coefficient or the penalty intensity factor, the weights weight coefficient is the coefficient in the regression equation, and a curve of different color represents a different component of the weight coefficient vector. Alpha is larger to the left, and tends to zero to the right, so the weight coefficient can be constrained by α.

Find the best hyperparameter α by cross-verification

Cross-validation Cross Validation divides the training set into several parts, which are used for training, testing and verification respectively, in order to find the best super parameters. After setting a set of alpha in the program, the program can verify itself and return the best alpha.

This is the end of the article on "regularization of loss function in big data". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.