Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to use the FocalLoss function in Tensorflow. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

1. FocalLoss introduction

FocalLoss is an improvement on the basis of cross-entropy function, which is mainly improved in two places.

(1) the first point of improvement is shown in the following formula.

First of all, a weight factor is added to the original cross-entropy function, in which gamma > 0, which makes us pay more attention to the difficult and misclassified samples. For example, if gamma = 2, for normal samples, if the prediction result is 0.97, then it must be easy to classify samples, the weight value is 0.0009, the loss function value will be very small; for positive samples, if the prediction result is 0.3, then it must be difficult to classify samples, the weight value is 0.49, the loss function value will be relatively large. For negative samples, if the prediction result is 0.8, then it must be difficult to classify samples, the weight value is 0.64, the loss function value will be relatively large; for negative samples, if the prediction result is 0.1, then it must be easy to classify samples, the weight value is 0.01, the loss function value will be very small. However, when the prediction probability is 0.5, the value of loss function is only reduced by 0.25 times, so FocalLoss reduces the influence of simple samples and pays more attention to indistinguishable samples.

(2) the second point of improvement is shown in the following formula.

The following is a brief derivation of the function expression of FocalLoss function in binary classification.

The FocalLoss function can represent something like the following formula:

3. FocalLoss code implementation

The pseudo code of the expression FocalLoss derived above can be expressed as:

From here, you can see that the 1-y_pred item may be 0 or 1, which will cause the NAN phenomenon in the value of the log function, so it is necessary to truncate the y_pred item with a fixed range of values. Finally, the function is implemented under TensorFlow1.8.

Import tensorflow as tf

Def focal_loss (y_true, y_pred, alpha=0.25, gamma=2): epsilon = 1e-5 y_pred = tf.clip_by_value (y_pred, epsilon, 1-epsilon) logits = tf.log (y_pred / (1-y_pred)) weight_a = alpha * tf.pow ((1-y_pred), gamma) * y_true weight_b = (1-alpha) * tf.pow (y_pred) Gamma) * (1-y_true) loss = tf.log1p (tf.exp (- logits)) * (weight_a + weight_b) + logits * weight_b return tf.reduce_mean (loss) about how to use the FocalLoss function in Tensorflow is shared here. I hope the above content can be of some help to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.