Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to operate ReLU and Pooling in CNN". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to operate ReLU and Pooling in CNN.

02

-

ReLU operation

The activation function used by CNN is not the Sigmoid function. We all know that the biggest problem with the Sigmoid function is that the gradient disappears with deep learning, which will cause the final convergence speed to become very slow. It has been proved that the performance will be better by adopting another function. This is the ReLU function, which looks like this:

It can be seen that when x is greater than 0, it is a linear function, and when x is less than 0, it is a nonlinear function, so ReLu is regarded as a nonlinear function.

The practical significance of ReLU function to CNN

You can see that the convolution operation in CNN is a linear operation, the corresponding elements are multiplied, and then summed. We also know that in the real world, most of the data are nonlinear, so it is necessary to introduce a nonlinear activation function. Let's look at the impact on the image through ReLU operation:

Original image

The picture on the left is the image after convolution of the original image, and the picture on the right is the image after ReLU. For the single-channel map, the range of values is 0mur255, of which 255 is white and 0 is black, because the result after convolution may be negative, so after ReLU, all values are non-negative, and there are no more negative values in question.

03

-

Pooling layer

The Pooling layer can reduce the dimension of the input features of the upper layer, but at the same time maintain its most important information. There are many kinds of Pooling operations: maximum pooling, average pooling, summing pooling and so on.

Take the maximum pooling as an example. After the ReLU operation, pooling first defines an adjacent region, then finds the maximum value of this region, and then selects a step size to traverse the image in turn, as shown in the following figure:

The practical significance of Pooling Operation

It reduces the feature dimension of the input and makes it easier to manage.

With the first item, it will also reduce the number of weight parameters and make it easier to converge.

The Pooling operation is equivalent to the pruning operation in the decision tree and can prevent overfitting.

After pooling operation, CNN can adapt to small displacement, distortion and displacement of the picture.

After the eating operation, CNN can help us to achieve invariance on a certain scale, which is very important because the image is no longer disturbed by the external environment.

04

-

CNN summary

Now that we have introduced the core technology used by CNN, if we combine the convolution operation, ReLU operation and Pooling operation, we can get the following depth network. We can see that the convolution and pooling have been carried out twice to complete the feature extraction of the input image, followed by the full connection layer, which is consistent with the DNN (fully connected multi-layer neural network) mentioned earlier. You can refer to the previous push to review the multilayer perceptrons.

After the full connection layer, all the main features extracted from the previous layer are used, the general mean variance is used as the loss function, and the softmax classifier can be used to complete the multi-classification task in the output layer. It is reasonable to see that the predicted result is a ship.

Here is a summary of CNN's algorithm:

Initialize all filter cores, hyperparameters, weight parameters, etc.

Input the trained image data through forward propagation, including convolution operation, ReLU operation, pooling operation and forward propagation of the full connection layer, and finally get the probability value belonging to each class in the output layer.

Calculate the total error at the output layer, using the mean square error term: total error = ∑ (target probability-output probability) ²

The gradient of the error term relative to the weight parameter and offset term is calculated by back propagation, so as to minimize the error sum in 3, and stop training when it reaches a certain threshold.

Repeat the 2-4 operation for all image data.

At this point, I believe you have a deeper understanding of "how ReLU and Pooling operate in CNN". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.