Shulou(Shulou.com)06/02 Report--

How to understand Python LeNet network and pytorch implementation, I believe that many inexperienced people are helpless about this, this article summarizes the causes and solutions of the problem, through this article I hope you can solve this problem.

1. Introduction to LeNet

LeNet neural networks were proposed by Yan LeCun, one of the Big Three of Deep Learning, who is also the father of convolutional neural networks (CNN). LeNet is mainly used for handwritten character recognition and classification, and has been put into use in American banks. The implementation of LeNet established the structure of CNN, and now many of the contents of neural networks can be seen in the network structure of LeNet, such as convolutional layers, Pooling layers, ReLU layers. Although LeNet was proposed as early as the 1990s, due to the lack of large-scale training data and the low performance of computer hardware at that time, LeNet neural networks were not ideal for dealing with complex problems. Although LeNet network structure is relatively simple, but just suitable for neural network entry learning.

LetNet network model

LeNet network model is generally referred to as LeNet-5. I believe everyone must have seen this picture when learning this model.

This picture is also a model picture in the original paper, which may seem a bit uncomfortable. The following picture is a network model picture I made on drawio software, as follows:

Correction, the calculation after the second Conv2d layer in the above figure should be 10, written as 5.

I believe that friends who have learned the basics of convolutional neural networks should be able to clearly understand this figure. For the calculation on the right, the formula is also given in the upper left corner of the figure. The input shape and output shape of each layer in the figure above are written in detail for everyone. For the calculation formula and the approximate structure of the model, see the following figure.(It is recommended to look at the corresponding upper and lower figures together for easier understanding.)

LeNet-5 network model simply includes convolutional layer, maximum pooling layer, full connection layer and relu, softmax activation function, input image size in the model and the number of convolution kernels in each layer, step size is determined by the model, generally do not modify at will, what can be changed is the final output result, that is, the number of classes (num_classes). Flatten operation is also called flattening operation. We all know that the input to the fully connected layer is a feature and a one-dimensional vector, but the feature matrix extracted from the convolutional network feature is not one-dimensional and needs to be sent to the fully connected layer, so we need to flatten it into one-dimensional operation.

3.Pytorch implementation LeNet

Python code is as follows

from torch import nnimport torchimport torch.nn.functional as F''' Description: LeNet is a 5-layer network. 2.nn.ReLU(inplace=True) parameter is True to modify the tensors passed down from the upper network Conv2d directly, which can save computing memory and save more storage of other variables. 3. The dimension annotation of this model omits the size of N(batch_size), i.e. input(3, 32, 32)-->input(N, 3, 32, 32) 4.nn.init.xavier_uniform_(m.weight) Fill the input tensor or variable with a uniform distribution of generated values, and the resulting tensor samples values from U(-a, a), where a = gain * sqrt( 2/(fan_in + fan_out))* sqrt(3), gain is an optional scaling factor, the default is 1 'fan_in' retains the magnitude of weight variance in forward propagation,'fan_out' retains the magnitude in reverse propagation 5.nn.init.constant_(m.bias, 0) Fill all dimension tensors with a constant 0''class LeNet(nn.Module): def __init__(self, num_classes=10, init_weights=False): super(LeNet, self).__ init__() self.conv1 = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=5, stride=1) self.maxpool1 = nn.MaxPool2d(kernel_size=2, stride=2) self.relu = nn.ReLU(True) self.conv2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, stride=1) self.maxpool2 = nn.MaxPool2d(kernel_size=2, stride=2) self.fc1 = nn.Linear(32 * 5 * 5, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84, num_classes) if init_weights: self._ initialize_weights() def forward(self, x): x = self.conv1(x) # input(3, 32, 32) output(16, 28, 28) x = self.relu(x) #Activation function x = self.maxpool1(x) # output(16, 14, 14) x = self.conv2(x) # output(32, 10, 10) x = self.relu(x) #Activation function x = self.maxpool2(x) # output(32, 5, 5) x = torch.flatten(x, start_dim=1) # output(32*5*5) N generation table batch_size x = self.fc1(x) # output(120) x = self.relu(x) #Activation function x = self.fc2(x) # output(84) x = self.relu(x) #Activation function x = self.fc3(x) # output(num_classes) return x def _initialize_weights(self): for m in self.modules(): if isinstance(m, nn.Conv2d): nn.init.xavier_uniform_(m.weight) if m.bias is not None: nn.init.constant_(m.bias, 0) elif isinstance(m, nn.Linear): nn.init.xavier_uniform_(m.weight) nn.init.constant_(m.bias, 0)

or

The following one does not initialize its own weights and biases, and uses the default initialization method

import torch.nn as nnimport torch.nn.functional as Fimport torchclass LeNet(nn.Module): def __init__(self): super(LeNet, self).__ init__() self.conv1 = nn.Conv2d(3, 16, 5) self.pool1 = nn.MaxPool2d(2, 2) self.conv2 = nn.Conv2d(16, 32, 5) self.pool2 = nn.MaxPool2d(2, 2) self.fc1 = nn.Linear(32*5*5, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84, 10) def forward(self, x): x = F.relu(self.conv1(x)) # output(16, 28, 28) x = self.pool1(x) # output(16, 14, 14) x = F.relu(self.conv2(x)) # output(32, 10, 10) x = self.pool2(x) # output(32, 5, 5) x = x.view(x.size(0), -1) # output(32*5*5) x = F.relu(self.fc1(x)) # output(120) x = F.relu(self.fc2(x)) # output(84) x = self.fc3(x) # output(10) return x

nn.Linear is a fully connected layer. Except for the last fully connected layer, all other layers need relu activation. By default, there is no padding operation.

The order of parameters corresponding to nn.Conv2d must be remembered:

1.in_channels: Number of channels or depth entered

2.out_channels: Number or depth of output channels

kernel_size: Size of convolution kernel

4.stride: stride size, default 1

5. padding:padding size, default 0

6.dilation: dilation size, default 1, temporarily unavailable

7.group Number of grouping groups, default 1

8.bias: default True, Boolean, whether to use bias value

9.padding_mode: 0 by default

It doesn't matter if you can't remember the parameter order, but you need to remember the parameter name.

After reading the above, do you know how to understand Python LeNet network and pytorch implementation? If you still want to learn more skills or want to know more related content, welcome to pay attention to the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.