Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to carry out the loss function losses analysis, the content of the article is of high quality, so the editor will share it for you as a reference. I hope you will have some understanding of the relevant knowledge after reading this article.

First, an overview of loss function

Generally speaking, the objective function of supervised learning consists of loss function and regularization term. (Objective = Loss + Regularization)

For keras model, the regularization term in the objective function is generally specified in each layer. For example, parameters such as kernel_regularizer and bias_regularizer of Dense are used to specify weights using L1 or L2 regularization terms. In addition, parameters such as kernel_constraint and bias_constraint can also be used to restrict the range of weights, which is also a regularization method.

The loss function is specified at model compilation time. For the regression model, the commonly used loss function is the square loss function mean_squared_error.

For the binary classification model, the bivariate cross-entropy loss function binary_crossentropy is usually used.

For the multi-classification model, if the label is coded by the category number, the class cross-entropy loss function categorical_crossentropy is used. If label encodes one-hot, it needs to use the sparse class cross-entropy loss function sparse_categorical_crossentropy.

If necessary, you can also customize the loss function. The custom loss function needs to receive two tensors, which are called as input parameters, and output a scalar as the value of the loss function.

Second, loss function and regularization term

For the keras model, the regularization items in the objective function are generally specified in each layer, and the loss function is specified at model compilation time.

Third, built-in loss function

The built-in loss function generally has two forms: class implementation and function implementation.

For example, both CategoricalCrossentropy and categorical_crossentropy are class cross-entropy loss functions, the former is the implementation form of the class, and the latter is the implementation form of the function.

Some commonly used built-in loss functions are described below.

Mean_squared_error (square difference error loss, for regression, abbreviated as mse, class implementation is MeanSquaredError and MSE)

Mean_absolute_error (absolute error loss, for regression, abbreviated as mae, class implementation as MeanAbsoluteError and MAE)

Mean_absolute_percentage_error (average percentage error loss, for regression, abbreviated as mape, class implementation as MeanAbsolutePercentageError and MAPE)

Huber (Huber loss, only class implementation, for regression, between mse and mae, robust to outliers, has some advantages over mse)

Binary_crossentropy (binary Cross Entropy, for binary classification, class implementation is BinaryCrossentropy)

Categorical_crossentropy (Category Cross Entropy, for multi-classification, requires label to be onehot encoded, class implemented as CategoricalCrossentropy)

Sparse_categorical_crossentropy (sparse Category Cross Entropy, used for multiple classifications, requires label to be numbered and class implemented to be SparseCategoricalCrossentropy)

Hinge (hinge loss function, for binary classification, the most famous application is as the loss function of support vector machine SVM, class implementation form is Hinge)

Kld (relative entropy loss, also known as KL divergence) is often used as the loss function of the maximum expectation algorithm EM, an information measure of the difference between the two probability distributions. Class implementation is KLDivergence or KLD)

Cosine_similarity (cosine similarity, can be used for multi-classification, class implementation form is CosineSimilarity)

Third, customize the loss function

The custom loss function takes two tensors as input parameters and outputs a scalar as the value of the loss function.

The tf.keras.losses.Loss can also be subclassed and the call method can be rewritten to realize the loss calculation logic, thus the implementation of the class of the loss function can be obtained.

Here is a demonstration of a custom implementation of Focal Loss. Focal Loss is an improved form of loss function for binary_crossentropy.

In the case of category imbalance and difficult training samples, it can achieve better results than binary cross-entropy.

For more information, see "how to evaluate Kaiming's Focal Loss for Dense Object Detection?" "

Https://www.zhihu.com/question/63581984

On how to carry out the loss function losses analysis is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.