Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what is the method of gradient lifting decision tree GBDT". In daily operation, I believe that many people have doubts about the method of gradient lifting decision tree GBDT. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the questions of "what is the method of gradient lifting decision tree GBDT?" Next, please follow the editor to study!

G-B-D-T gradient lifting decision tree, as its name implies, is a machine learning model related to gradient and lifting the decision tree. We might as well talk about the attributives GBD from back to front in order to understand the essence of the model.

DT (Decision Tree) decision tree. T needless to say, as a common data structure appears in a variety of algorithms. DT decision tree, there are classification tree and regression tree, the previous article talked about the classification tree, see and. The principle and mechanism of the regression tree is similar to that of the classification tree, except that the classification tree only returns the unique classification at the leaf node, while each node of the regression tree can return the predicted value, which is usually the mean of all samples under the current node.

B (Boosting) promotion. That is to say, to further improve on the basis of the original model, the basic idea of improving the decision tree BDT is to use multiple decision trees for serial modeling. The specific process is that, for each decision tree after the first tree, the secondary modeling is based on the output of the previous decision tree, and the whole serial modeling process is equivalent to modifying the prediction results to the target value.

G (Gradient) gradient. The size of the gradient reflects the distance between the current prediction and the target value. Therefore, in the serial decision tree model described in B above, except for the first decision tree using the original prediction index, each subsequent decision tree is built with a negative gradient (which can be understood as residual or increment) calculated by the predicted value and the target value of the previous decision tree. This is equivalent to weighting and classifying the misclassified samples many times, so that the final residual of the samples approaches 0. Except for the other trees of the first tree, because the residual or increment of the target is modeled and predicted, the GBDT model only needs to accumulate the output of each decision tree in the process to get the final prediction output.

Here is an example of predicting age, which is widely circulated on the Internet to describe the process of GBDT. A total of four samples:

A (high consumption, often asked questions by his younger brother) 27 years old

B (high consumption, often ask seniors questions) 23 years old

C (low consumption, often asked questions by his younger brother) 17 years old

D (low consumption, often ask seniors questions) 13 years old

The serial decision tree is built as shown in the following figure. According to the fitting of the model, the ages of A, B, C and D are 25 + 2 = 27 years, 25-2 = 23 years, 15 + 2 = 17 years and 15-2 = 13 years respectively.

To sum up, the core idea of GBDT is to train n (n > 2) decision trees serially, in which the I (1 < I ≤ n) tree learns the negative gradient of the I-1 tree (can be understood as residual or increment), and the output results of n trees are accumulated as the final output.

Advantages and limitations of GBDT

Advantages:

1. In the prediction stage, the calculation is fast, and the calculation can be parallelized between trees.

two。 In the densely distributed dataset, the generalization ability and expression ability are very good.

3. Using the decision tree as the weak classifier makes the GBDT model have good interpretation and robustness, can automatically find the high-order relationship between features, and does not need special data preprocessing.

Limitations:

1.GBDT does not perform as well as SVM or neural network on high-dimensional and sparse data sets.

Compared with other models, the advantage of 2.GBDT in dealing with text classification features is not as obvious as when dealing with numerical features.

3. The training process needs serial training, and only some local parallel methods can be used to improve the training speed in the decision tree.

At this point, the study of "what is the method of gradient lifting decision tree GBDT" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.