Shulou(Shulou.com)06/01 Report--

How to analyze the knowledge points in PyTorch, I believe that many inexperienced people do not know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Summary:

By convention, the tensor of all attributes requires_grad=False is the leaf node (i.e., leaf tensor, leaf node tensor). For the attribute requires_grad=True, the tensor may be the leaf node tensor or not the leaf node tensor but the intermediate node tensor. If the tensor's attribute requires_grad=True is used to create directly, that is, its attribute grad_fn=None, then it is the leaf node. If the attribute requires_grad=True of the tensor is not directly created by the user, but is generated by other tensors through certain operations, then it is not a leaf tensor, but an intermediate node tensor, and its attribute grad_fn is not None, for example: grad_fn=, this means that the tensor is generated through the torch.mean () operation, is an intermediate result, so it is an intermediate node tensor So it's not a leaf node tensor. To determine whether a tensor is a leaf node, you can look at it through its attribute is_leaf. The attribute requires_grad of a tensor is used to indicate whether a gradient needs to be calculated for the tensor during back propagation. If the property of the tensor is requires_grad=False, then there is no need to calculate the gradient for the tensor, and there is no need to optimize the learning for the tensor. In the operation of PyTorch, if the attribute requires_grad of all the input tensors participating in the operation is False, then the result of the operation is that the attribute requires_grad of the output tensor is also False, otherwise it is True. That is, as long as one of the input tensors requires a gradient (attribute requires_grad=True), then the resulting tensor also requires a gradient (attribute requires_grad=True). Only when all the input tensors do not require a gradient, the resulting tensor does not need a gradient. For the tensor of attribute requires_grad=True, the gradient is calculated for the tensor during back propagation. However, the automatic gradient mechanism of pytorch does not save the gradient for the intermediate result, that is, only the gradient calculated for the leaf node is saved in the attribute grad of the leaf node tensor, and the gradient of the tensor is not saved in the attribute grad of the intermediate node tensor, which is for the consideration of efficiency. The attribute grad of the intermediate node tensor is None. If you need to save the gradient for the intermediate node, you can have the intermediate node call the method retain_grad (), so that the gradient will be saved in the grad attribute of the intermediate node. After reading the above, have you mastered the method of how to analyze the knowledge points in PyTorch? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.