Shulou(Shulou.com)06/01 Report--

This article will explain in detail how pytorch prints the network return gradient. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Demand:

Print gradients and check network learning

Net = your_network () .cuda () def train ():... Outputs = net (inputs) loss = criterion (outputs, targets) loss.backward () for name, parms in net.named_parameters (): print ('- > name:', name,'--> grad_requirs:',parms.requires_grad,'--> grad_value:',parms.grad). Printing the result:

Name indicates the name of the network parameter; parms.requires_grad indicates whether the parameter is learnable and frozen; and parm.grad prints the gradient value of the parameter.

Supplement: gradient calculation of pytorch

Look at the code ~ import torchfrom torch.autograd import Variablex = torch.Tensor ([[1.meme2.pence3.], [4.meme 5.pence6.]) # grad_fn is Nonex = Variable (x, requires_grad=True) y = x + 2z = y*y*3out = z.mean () # x-> y-> z-> outprint (x) print (y) print (z) print (out) # result: tensor ([1.mem2.pence3.] Requires_grad=True) tensor ([3, 4, 5.], [6, 7, 8.], grad_fn=) tensor ([27, 48, 75.], [108., 147. 192.], grad_fn=) tensor (99.5000, grad_fn=)

If the result variable for the derivation of graph leaves is a scalar, then gradient defaults to None, or specifies "torch.Tensor ([1.0])".

If the result variable for the derivation of graph leaves is a vector, then gradient cannot be defaulted. If tensor is at the same latitude as that vector,

Out.backward () print (x.grad) # result: tensor ([3, 4, 5.], [6, 7, 8.]) # if z is derived with respect to x, the gradient parameter must be specified: gradients = torch.Tensor ([2. Rect 1. 1.], [1. Rect 1.]) z.backward (gradient=gradients) # if z is not a scalar Then we first construct a scalar value: l = torch.sum (z*gradient), and then calculate the gradient of L to each leaf Variable # x to L. x.grad# result: tensor ([36.,24.,30.], [36.,42.48.]) Error condition z.backward () print (x.grad) # error: RuntimeError: grad can be implicitly created only for scalar outputs can only create an implicit variable x1 = Variable for scalars (torch.Tensor ([[1. X2 = Variable (torch.arange (4) .view (2jue 2) .type (torch.float), requires_grad=True) c = x2.mm (x1) c.backward (torch.ones_like (c)) # c.backward () # RuntimeError: grad can be implicitly created only for scalar outputsprint (x2.grad)

From the above example, out is a constant, so you can create hidden variables by default. If the backpropagation is not a constant, you should know that the specific value of the matrix is the loss matrix in the network. In the process of directional propagation, the normalized loss is multiplied by the gradient to update the parameters of each neuron.

See a blog saying: loss = criterion (outputs, labels) corresponds to loss + = (label [k]-h) * (label [k]-h) / 2

Just ask for loss (in fact, I think we don't have to do this step. We don't need the loss value for back propagation, just to let us know what the current loss is).

I think it must be the specific value of loss in order to compare the threshold for classification, through the nonlinear activation function to determine whether to activate or not.

This is the end of the article on "how to print the network return gradient in pytorch". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.