Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to train RNN through PPT. Many people may not know much about it. In order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

RNN is an important language model (language model), its input is not limited by the number of input words, it can be of any length, and the weights of the hidden layer nodes are equal. RNN as a neural network model, its training technology still uses the back propagation method. Next, through the Stanford PPT, the whole training process of solving the weight parameter w is described in detail.

The loss function chooses cross entropy, and many algorithm models of machine learning also choose cross entropy. Obviously, the size of the loss function does not want to be affected by the number of words in the training corpus, so we can calculate the cross-entropy of all the training words and then average them.

Yj is the correct word, YJ ^ is the predicted word, V is the total number of words, T is the total number of training samples, as shown in the following figure, the correct output of the input word the should be students, corresponding to the loss of J1.

Y2,..., yn are calculated in turn, and the average of all losses is obtained.

The value of the parameter is adjusted by back propagation, as shown below, this is a time back propagation algorithm, and the loss gradient of t-time step is the sum of the loss gradient of the previous t-step.

At this point, the formula for adjusting the weight has been obtained.

After reading the above, do you have any further understanding of how to train RNN through PPT? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.