Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what is the meaning of batch, epoch and iteration". In daily operation, I believe that many people have doubts about the meaning of batch, epoch and iteration. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "what is the meaning of batch, epoch and iteration?" Next, please follow the editor to study!

First, the loss function is calculated by traversing all the data sets, and then the gradient of the function to each parameter is calculated to update the gradient. This method has to look at all the samples in the data set every time the parameters are updated, which requires a large amount of computation, the calculation speed is slow, and does not support online learning, which is called Batch gradient descent, and the batch gradient drops.

On the other hand, each look at the data is calculated as the loss function, and then the gradient update parameters are calculated, which is called random gradient descent, stochastic gradient descent. The speed of this method is relatively fast, but the convergence performance is not very good, it may dangle around the best point, and the hit is less than the best point. The updates of the two parameters may also cancel each other out, causing the objective function to fluctuate violently.

In order to overcome the shortcomings of the two methods, now generally use a compromise means, mini-batch gradient decent, a small batch of gradient decline, this method divides the data into several batches, according to the batch to update the parameters, so that a group of data in a batch jointly determines the direction of the gradient, falling is not easy to run off, reducing randomness. On the other hand, because the sample size of the batch is much smaller than that of the whole data set, the amount of calculation is not very large.

The optimizer SGD used today is an acronym for stochastic gradient descent, but that doesn't mean it's updated once a sample, or based on mini-batch.

What does batch epoch iteration stand for?

(1) batchsize: batch size. In deep learning, SGD training is generally used, that is, batchsize samples are taken from the training set for each training.

(2) iteration:1 iteration is equal to one training with batchsize samples.

(3) epoch:1 epoch equals to training once with all the samples in the training set. Generally speaking, the value of epoch means that the whole data set is turned several times.

For example, if the training set has 500 samples, batchsize = 10, then the whole sample set is trained: iteration=50,epoch=1.

Batch: in-depth learning the loss function required for each parameter update is not obtained by a single data, but by a set of data weighted, the number of which is batchsize.

The maximum of batchsize is the total number of samples N, and the minimum of Full batch learning; is 1, that is, only one sample is trained at a time, which is called online learning (Online Learning). When we study in batches, we use all the training data to complete one Forword operation and one BP operation at a time, which becomes an epoch.

The usage of tf.clip_by_value

Tf.clip_by_value (A, min, max): enter a tensor A to compress the value of each element in A between min and max. Make it equal to min if it is less than min, and equal to max if it is greater than max.

Import tensorflow as tf; import numpy as np; A = np.array with tf.Session () as sess: print sess.run (tf.clip_by_value (A, 2, 5)) output: [[22 24] [3 455]], the study on "what is the meaning of batch, epoch and iteration" is over, hoping to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.