Shulou(Shulou.com)06/01 Report--

Introduction to Machine Learning (Machine Learning) and decision Tree (Decision Tree)

The introduction series of machine learning are some records and experiences in the process of personal learning. It is mainly presented in the form of key points, concise and clear.

1. What is machine learning?

A more general understanding is to predict the future based on existing data.

two。 Core idea: Generalization

It can be understood as induction and generalization. Just like human learning, find out the connection between one thing and another.

3. Inductive machine learning (Inductive machine learning)

Its core idea is to use the training data and find out a set of formula f which is suitable for reality. And then use this formula to predict the results in reality.

It is divided into four types.

Regression regression type: mainly suitable for predicting real data

Binary Classification: classification of Yes and No

Multiple Classification: an one-to-many classification problem

Ranking: the ranking problem of relevance

4. The first model: decision tree

The core idea of decision tree: divide and conquer. I believe you are already familiar with this concept when you learn recursive programming (recursion), which is to split the problem into simple enough, and then solve it.

In order to transform real-world problems into models that we can use, we propose several concepts here.

Feature (feature): the question to be asked every time the decision tree makes a decision

Feature value (property value): simply understood as the answer to a question

Training data: training data > > is a collection of many examples. Each example usually appears in the form of (xPowery), where x is the input data and y is the actual result.

5. The working principle of decision Tree

For example, the following decision tree virtual code is taken from "A Course in Machine Learning" (Hal)

The core idea is to choose the most useful one from the remaining feature to ask questions, and if the answer can directly determine the final result, then jump out of recursion. If there is no feature left to ask, then return to the default answer. Otherwise, the whole data is divided into two groups, namely yes combined no group, and then divided into two branches to continue recursion (line 16 and 17).

6.Loss Function,L (y, y *)

Y is the real result, and y * is the result of machine prediction L represents the loss function. So loss function is used to quantify errors.

According to different types, it can be divided into:

Regression regression class:

Square loss: square loss: l (y, y *) = (y-y *) 2

Absolute loss: absolute loss: l (y, y *) = | y-y * |

Binary yes or no questions and multiple groups of classification questions:

L (y, y *) = 0, if y = y * (zero loss)

L (y, y *) = 1, if y y *

7. Average loss Expected Loss

Expected loss = Sum (xpeny) (D (xperiy) * L (y, y*))

D (x ~ ()) is the probability distribution of (x ~ () ~ ()), which can be understood as (x _ ()). So it's the general algorithm of the expected value, multiplying the probability by the loss and then summing.

So for the equation f derived from our machine learning, the lower the Expected Loss, the better.

8.Inductive Bias inductive bias

The choice of different models or methods will lead to different biases. Decision trees, for example, are better at dealing with individual problems, but have little effect on several feature interactions.

We will continue to introduce the biases of different models in subsequent articles, which are mentioned here first.

9. Shallow Decision Tree

That is, the decision tree with the maximum depth, we can use a depth d to limit the number of the deepest layers of the decision tree.

10. Underfitting and Overfitting

These are two very important concepts, Underfitting refers to, do not learn enough, for example, students do not learn anything to take the exam, naturally will not do well.

Overfitting means that when students become nerds, they will only learn by rote, and they will not be able to do the exam by changing a few numbers.

In the decision tree, if a feature doesn't ask, it's underfitting, learning nothing but returning most of the answers in the training data at a time.

If you ask a lot of feature and ask all the feature aside, it is possible to overfitting, memorizing according to the training data rather than inductive learning.

11. Data allocation

Data description percentage Training Data training data learn and summarize equation f

70%Development Data develops data tuning Super parameters (Hyperparameters) to avoid underfitting and overfitting. 10%Test Data test data are used to evaluate the final learning effect. Never peek at the test data! Peeking at the test data is like peeking at the exam paper, which will only lead to targeted programming. But once you get to the real operating environment, the effect will be 108000 miles. 20%

Super parameters (Hyperparameters) are parameters that can affect normal parameters. For example, the maximum query depth d is a super parameter.

twelve。 General methods of machine learning

Divide the data into 70% training data, 10% R & D data, and 20% test data.

All hyperparameter of for loop (super parameter)

Using training data, training model f

Calculate Expected Loss using R & D data (average loss)

From all the super parameters above, select the group with the lowest average loss as the final model

Use test data to evaluate the final model

Reference

Hal, Daum é III. A Course In Machine Learning. 2nd ed. Self-published, 2017. Print.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.