Shulou(Shulou.com)05/31 Report--

The editor would like to share with you the example analysis of underfitting and over-fitting in big data's development. I believe most people don't know much about it, so share this article for your reference. I hope you will gain a lot after reading this article. let's learn about it!

Preface

Here we first introduce a noun, model capacity: popularly speaking, the capacity or expressive ability of the model refers to the ability of the model to fit complex functions. When the capacity of the model is larger, the hypothetical space of the function is larger, and the more likely it is to find a function model whose function is closer to the real distribution.

Note: in convolutional neural networks, the model capacity is usually measured by the number of network layers and the number of parameters to be optimized.

Underfit and overfit

The main results are as follows: (1) when the capacity of the model is too large, the network model not only learns the modes of the training set data, but also learns the additional observation errors. As a result, the learned model performs better on the training set, but it does not perform well on the unseen samples, that is, the generalization ability is weak. We call this phenomenon Overfitting.

(2) when the capacity of the model is too small, the model can not learn the mode of the training set data well, resulting in poor performance on the training set and poor performance on the unseen samples. We call this phenomenon Underfitting.

Occam's razor principle

So in the process of in-depth learning, how to choose the appropriate model capacity?

Statistical learning theory is difficult to give the minimum capacity of neural network, but we can use Occam razor principle (Occam's razor) to guide the design of neural network (that is, the choice of model capacity).

Occam's razor principle is a problem-solving principle put forward by William of Occam, a logician and Franciscan friar of St. Franciscans in the 14th century. It is used in the process of neural network design, that is to say, if the two-layer neural network structure can well express the real model, then the three-layer neural network can also be well expressed. But we should give priority to using a simpler two-layer neural network, because it has fewer parameters, easier to train, and easier to obtain good generalization errors through fewer training samples.

Model design

Through the verification set, we can judge whether the network model is overfitted or underfitted, so as to provide a judgment basis for adjusting the capacity of the network model.

For neural networks, the number of layers and parameters of the network is a very important reference index of network capacity.

Therefore, in the process of model design, the general method to solve the problem of over (under) fitting is:

The main results are as follows: (1) when the model is found to be overfitted, it is necessary to reduce the capacity of the network. This can be achieved by reducing the number of layers of the network and reducing the scale of the number of network parameters in each layer.

(2) when the model is found to be underfitted, it is necessary to increase the capacity of the network. It can be achieved by increasing the number of layers of the network and reducing the network parameters of each layer.

In order to demonstrate the influence of network layers on network capacity, we have done a visual decision boundary classification experiment. The experimental results are as follows:

Experimental conclusion:

As shown in the figure, we can see that with the deepening of the number of layers of the network, the decision boundary of the learned model is closer and closer to the training sample, and the phenomenon of over-fitting occurs. For this task, the two-layer neural network can obtain good generalization ability, and the deeper layer neural network does not improve the performance, but overfits, the generalization ability becomes worse, and the computational cost is higher.

The above is all the contents of the article "example Analysis of under-fit and over-fit in big data's Development". Thank you for your reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.