Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what is the connection and difference between LR and SVM". In daily operation, I believe that many people have doubts about the connection and difference between LR and SVM. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the question of "what is the connection and difference between LR and SVM?" Next, please follow the editor to study!

contact

1. Both LR and SVM can deal with classification problems, and are generally used to deal with linear binary classification problems (multi-classification problems can be dealt with in the improved case).

(2) if the kernel function is not considered, both LR and SVM are linear classification algorithms, that is to say, their classification decision planes are linear.

To make it clear here, the original LR and SVM are linear classifiers, which is why no one usually asks you what is the difference between a decision tree and LR, what is the difference between a decision tree and SVM, what is the difference between a nonlinear classifier and a linear classifier?

3. Both LR and SVM are supervised learning algorithms.

4both LR and SVM are discriminant models.

The discriminant model generates a discriminant function (or prediction model) that represents P (Y | X), and the generated model first calculates the joint probability p (YMagi X) and then converts it into a conditional probability through Bayesian formula. To put it simply, when calculating the discriminant model, the joint probability will not be calculated, but when calculating the generated model, the joint probability must be calculated first. Or put it this way: the generation algorithm tries to find out exactly how the data is generated, and then classifies a signal. Based on your generation hypothesis, which category is most likely to generate this signal, this signal belongs to that category. The discriminant model does not care about how the data is generated, it only cares about the differences between signals, and then uses the differences to simply classify a given signal.

The common discriminant models are KNN, SVM and LR, and the common generation models are naive Bayesian and hidden Markov model.

5 LR and SVM are widely known and widely used in academia and industry.

After talking about the similarities between LR and SVM, do you also think it is necessary to compare them? And is it better to compare LR and SVM than to compare decision tree and LR, decision tree and SVM, naive Bayes and LR, naive Bayes and SVM?

Difference

1. LR is a parametric model and SVM is a non-parametric model.

2. From the point of view of the objective function, the logical regression adopts Logistical Loss,SVM and hinge loss. The purpose of these two loss functions is to increase the weight of the data points that have a greater impact on the classification and reduce the weight of the data points that have less relationship with the classification.

The way to deal with SVM is to consider only Support Vectors, that is, the few points most relevant to classification, to learn classifiers. Through nonlinear mapping, logical regression greatly reduces the weight of the points far away from the classification plane, and relatively increases the weight of the data points most related to the classification.

3. The logical regression model is relatively simple and easy to understand, especially in large-scale linear classification. However, the understanding and optimization of SVM is relatively complex. After SVM is transformed into a dual problem, the classification only needs to calculate the distance from a few support vectors. When solving nonlinear problems, support vector machine uses kernel function mechanism, while LR usually does not use kernel function method.

(4) the linear SVM depends on the distance measure expressed by the data, so it is necessary to normalization,LR the data first.

5. The loss function of SVM has its own regularity (1x2 in the loss function | | w | | ^ 2 items), which is why SVM is a structural risk minimization algorithm. LR must add a regular term to the loss function separately.

At this point, the study on "what is the connection and difference between LR and SVM" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.