Shulou(Shulou.com)06/01 Report--

This article is about L1 and L2 regularization in linear regression. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

The regularization methods of L1 and L2 in regression modeling are described.

When dealing with complex data, we tend to create complex models. It's not always good to be too complicated. Overly complex models are what we call "overfitting". They perform well in training data, but not well in invisible test data.

There is a way to adjust the overfitting of the loss function, and that is punishment. By punishing or "regularizing" the large coefficients in the loss function, we reduce some (or all) coefficients so that the model is insensitive to noise in the data.

The two popular regularization forms used in regression are L1, also known as Lasso regression, and L2, also known as Ridge regression. In linear regression, we use ordinary least square (OLS) to fit the data: we square the residual (the difference between the actual value and the predicted value) to get the mean square error (MSE). The least square error, or the least square, is the most suitable model.

Let's look at the cost function of simple linear regression:

For multivariate linear regression, the cost function should be like this. Is the number of predictors or variables.

Therefore, with the predictor (? As the number increases, so does the complexity of the model. To alleviate this situation, we have added some forms of punishment to this cost function. This will reduce the complexity of the model, help prevent overfitting, may eliminate variables, and even reduce multicollinearity in the data.

L2-Ridge regression

L2 or Ridge return, will? The penalty item is added to the square of the coefficient. ? Is a hyperparameter, which means that its value is freely defined. You can see it at the end of the cost function.

Plus? Punishment, huh? A cost function with a constrained coefficient and a large penalty coefficient.

L1-Lasso regression

L1 or Lasso regression is almost the same thing, except for one important detail-the size of the coefficient is not squared, it's just an absolute value.

Here, the last part of the cost function is? Some coefficients can be precisely set to zero, while others can be directly reduced to zero. When some coefficients change to 00:00, the effect of Lasso regression is particularly useful because it estimates the cost and selects the coefficient at the same time.

And most importantly, before any type of regularization, the data should be standardized to the same size, otherwise fines will treat some coefficients unfairly.

Thank you for reading! This is the end of this article on "L1 and L2 regularization in linear regression". I hope the above content can be of some help to you, so that you can learn more knowledge. If you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.