Shulou(Shulou.com)06/03 Report--

This article mainly shows you the "sample analysis of torchvision.models in PyTorch", which is easy to understand and clear. I hope it can help you solve your doubts. Let the editor lead you to study and study the "sample analysis of torchvision.models in PyTorch".

There is a very important and easy-to-use package in the PyTorch framework: torchvision, which is mainly composed of three subpackages: torchvision.datasets, torchvision.models, and torchvision.transforms.

Details of these three subpackages can be found on the official website:

Http://pytorch.org/docs/master/torchvision/index.html .

For more information, please see github:

Https://github.com/pytorch/vision/tree/master/torchvision .

Introduce torchvision.models. Torchvision.models package contains alexnet, densenet, inception, resnet, squeezenet, vgg and other common network structures, and provides a pre-training model, which can be simply called to read the network structure and pre-training model.

Examples of use:

Import torchvisionmodel = torchvision.models.resnet50 (pretrained=True)

This introduces the pre-training model of resnet50. If you only need the network structure and do not need to initialize with the parameters of the pre-training model, then it is:

Model = torchvision.models.resnet50 (pretrained=False)

The same is true if you want to import densenet models, such as densenet169, and do not need to be pre-trained models:

Model = torchvision.models.densenet169 (pretrained=False)

Because the pretrained parameter defaults to False, it is equivalent to:

Model = torchvision.models.densenet169 ()

However, for the sake of clarity of the code, it is best to add parameter assignments.

Next, take the import resnet50 as an example to introduce the specific import model when the source code. When running model = torchvision.models.resnet50 (pretrained=True), it is done through the resnet.py script under the models package. The source code is as follows:

The first step is to import the necessary libraries, where model_zoo is the package associated with importing the pre-training model, and the all variable defines the name of the function or class that can be import from the outside. That's why you can call it with torchvision.models.resnet50 () earlier. The dictionary model_urls is the download address for the pre-training model.

Import torch.nn as nnimport mathimport torch.utils.model_zoo as model_zoo__all__ = ['ResNet',' resnet18', 'resnet34',' resnet50', 'resnet101',' resnet152'] model_urls = {'resnet18':' https://download.pytorch.org/models/resnet18-5c106cde.pth', 'resnet34':' https://download.pytorch.org/models/resnet34-333f7ec4.pth', 'resnet50': 'https://download.pytorch.org/models/resnet50-19c8e357.pth',' resnet101': 'https://download.pytorch.org/models/resnet101-5d3b4d8f.pth',' resnet152': 'https://download.pytorch.org/models/resnet152-b121ed2d.pth',}

Then there is the function resnet50, and the parameter pretrained defaults to False. First of all, model = ResNet (Bottleneck, [3,4,6,3], * * kwargs) is to build the network structure, and Bottleneck is another class for building bottleneck. There are many repetitive substructures in the construction of the ResNet network structure. These substructures are built through the Bottleneck class, which will be described later. Then if the parameter pretrained is True, the corresponding pre-training model is downloaded or imported according to the model_urls dictionary through the load_url function in model_zoo.py. Finally, initialize the network structure you build with the pre-trained model parameters by calling model's load_state_dict method, which is the common operation in PyTorch to initialize the layer of one model with the parameters of another model. Another important parameter of the load_state_dict method is strict, which defaults to True, which means that the layer of the pre-training model corresponds strictly to your network structure layer (such as layer name and dimension).

Def resnet50 (pretrained=False, * * kwargs): "Constructs a ResNet-50 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet" model = ResNet (Bottleneck, [3,4,6,3], * * kwargs) if pretrained: model.load_state_dict (model_urls ['resnet50']) return model

Other resnet18, resnet101 and other functions are basically similar to resnet50, but the main differences are as follows:

1. When building a network structure, the parameters of block are different, such as [2, 2, 2, 2] in resnet18 and [3, 4, 23, 3] in resnet101.

2. Different block classes are called. For example, Bottleneck classes are called in resnet50, resnet101 and resnet152, while BasicBlock classes are called in resnet18 and resnet34. The difference between these two classes is mainly due to the different number of convolution layers in the residual results, which is related to the network structure, which will be described in more detail later.

3. If you download the pre-training model, the keys of the model_urls dictionary are different, corresponding to different pre-training models. So let's take a look at how to build the network structure and how to import the pre-training model.

Def resnet18 (pretrained=False, * * kwargs): "Constructs a ResNet-18 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet "" model = ResNet (BasicBlock, [2,2,2,2], * * kwargs) if pretrained: model.load_state_dict (model_zoo.load_url (model_urls ['resnet18']) return modeldef resnet101 (pretrained=False, * * kwargs): "" Constructs a ResNet-101 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet "" model = ResNet (Bottleneck, [3,4,23,3], * * kwargs) if pretrained: model.load_state_dict (model_zoo.load_url (model_urls ['resnet101'])) return model

The ResNet network is built through the class ResNet. First of all, it inherits the base class of the network in PyTorch: torch.nn.Module, and the second is to rewrite the initialization _ _ init__ and forward methods. The main thing in initializing _ _ init__ is to define some layer parameters. The main purpose of forward method is to define the flow order of data between layers, that is, the connection order of layers. In addition, you can define other private methods in the class to modularize some operations, for example, the _ make_layer method here is used to build the four blocks in the ResNet network. The first input block of the _ make_layer method is the Bottleneck or BasicBlock class, the second input is the output channel of the blocks, and the third input is how many residual substructures are contained in each blocks, so the list of layers is [3, 4, 6, 3] of the previous resnet50.

The two important lines of code in the _ make_layer method are: 1. Layers.append (block (self.inplanes, planes, stride, downsample)), which stores the first residual structure of each blocks in the layers list. 2. For i in range (1, blocks): layers.append (block (self.inplanes, planes)), which stores the remaining residual structure of each blocks in the layers list, thus completing the construction of a blocks. Each residual is built through the Bottleneck class in both lines of code, followed by the Bottleneck class.

Class ResNet (nn.Module): def _ init__ (self, block, layers, num_classes=1000): self.inplanes = 64 super (ResNet, self). _ _ init__ () self.conv1 = nn.Conv2d (3,64, kernel_size=7, stride=2, padding=3, bias=False) self.bn1 = nn.BatchNorm2d (64) self.relu = nn.ReLU (inplace=True) self.maxpool = nn.MaxPool2d (kernel_size=3, stride=2, padding=1) self.layer1 = self._make_layer (block, 64 Layers [0]) self.layer2 = self._make_layer (block, 128, layers [1], stride=2) self.layer3 = self._make_layer (block, 256, layers [2], stride=2) self.layer4 = self._make_layer (block, 512, layers [3], stride=2) self.avgpool = nn.AvgPool2d (7, stride=1) self.fc = nn.Linear (512 * block.expansion, num_classes) for m in self.modules (): if isinstance (m) Nn.Conv2d): n = m.kernel_size [0] * m.kernel_size [1] * m.out_channels m.weight.data.elif isinstance _ (0, math.sqrt (2. / n)) elif isinstance (m, nn.BatchNorm2d): m.weight.data.fill1 m.bias.data.zero.def _ make_layer (self, block, planes, blocks Stride=1): downsample = None if stride! = 1 or self.inplanes! = planes * block.expansion: downsample = nn.Sequential (nn.Conv2d (self.inplanes, planes * block.expansion, kernel_size=1, stride=stride, bias=False), nn.BatchNorm2d (planes * block.expansion), layers = [] layers.append (block (self.inplanes, planes, stride, downsample) self.inplanes = planes * block.expansion for i in range (1, blocks): layers.append (block (self.inplanes) Planes) return nn.Sequential (* layers) def forward (self, x): X = self.conv1 (x) x = self.bn1 (x) x = self.relu (x) x = self.maxpool (x) x = self.layer1 (x) x = self.layer2 (x) x = self.layer3 (x) x = self.layer4 (x) x = self.avgpool (x) x = x.view (x.size (0),-1) x = self.fc (x) return x

As you can see from the previous ResNet class, when constructing a ResNet network, the most important thing is the Bottleneck class, because ResNet is made up of residual structures, and the Bottleneck class completes the construction of the residual structure. Again, Bottlenect inherits the torch.nn.Module class and overrides the _ _ init__ and forward methods. As can be seen from the forward method, bottleneck is the three main convolution layer, BN layer, and activation layer that we are familiar with, and the final out + = residual is the operation of element-wise add.

Class Bottleneck (nn.Module): expansion = 4 def _ init__ (self, inplanes, planes, stride=1, downsample=None): super (Bottleneck, self). _ _ init__ () self.conv1 = nn.Conv2d (inplanes, planes, kernel_size=1, bias=False) self.bn1 = nn.BatchNorm2d (planes) self.conv2 = nn.Conv2d (planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) self.bn2 = nn.BatchNorm2d (planes) self.conv3 = nn.Conv2d (nn.Conv2d, planes * 4, planes Bias=False) self.bn3 = nn.BatchNorm2d (planes * 4) self.relu = nn.ReLU (inplace=True) self.downsample = downsample self.stride = stride def forward (self X): residual = x out = self.conv1 (x) out = self.bn1 (out) out = self.relu (out) out = self.conv2 (out) out = self.bn2 (out) out = self.relu (out) out = self.conv3 (out) out = self.bn3 (out) if self.downsample is not None: residual = self.downsample (x) out + = residual out = self.relu (out) return out

The BasicBlock class is similar to the Bottleneck class, the former is mainly used to build ResNet18 and ResNet34 networks, because the residual structure of these two networks contains only two convolution layers and does not have the concept of bottleneck in the Bottleneck class. So in this class, the first convolution layer uses the convolution of kernel_size=3, as shown in the conv3x3 function.

Def conv3x3 (in_planes, out_planes, stride=1): "" 3x3 convolution with padding "" return nn.Conv2d (in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False) class BasicBlock (nn.Module): expansion = 1 def _ init__ (self, inplanes, planes, stride=1, downsample=None): super (BasicBlock, self). _ _ init__ () self.conv1 = conv3x3 (inplanes, planes, stride) self.bn1 = nn.BatchNorm2d (planes) self.relu = nn.ReLU (nn.ReLU) inplace=True = inplace=True Planes) self.bn2 = nn.BatchNorm2d (planes) self.downsample = downsample self.stride = stride def forward (self, x): residual = x out = self.conv1 (x) out = self.bn1 (out) out = self.relu (out) out = self.conv2 (out) out = self.bn2 (out) if self.downsample is not None: residual = self.downsample (x) out + = residual out = self.relu (out) return out

After introducing how to build the network, the next step is how to obtain the pre-training model. I mentioned this line of code earlier: if pretrained: model.load_state_dict (model_zoo.load_url (model_urls ['resnet50'])), which mainly imports the corresponding pre-training model according to the model_urls dictionary through the load_url function in model_zoo.py, and the github address of the models_zoo.py script:

Https://github.com/pytorch/pytorch/blob/master/torch/utils/model_zoo.py .

The source code of load_url function is as follows.

First of all, model_dir is the save address of the downloaded model, if not specified, it will be saved in the .torch directory of the project, preferably specified. Cached_file is the path where the model is saved plus the model name. The next if not os.path.exists (cached_file) statement is used to determine whether the model already exists in the specified directory. If it already exists, call the torch.load API to import the model. If it does not exist, download it from the Internet. The download is done through _ download_url_to_file (url, cached_file, hash_prefix, progress=progress). The point is that the model is imported through the torch.load () interface, regardless of whether your model is downloaded from the Internet or is already locally available.

Def load_url (url, model_dir=None, map_location=None, progress=True): r "Loads the Torch serialized object at the given URL. If the object is already present in `model_ dir`, it's deserialized and returned. The filename part of the URL should follow the naming convention ``filename- .ext``where `is the first eight or more digits of the SHA256 hash of the contents of the file. The hash is used to ensure unique names and to verify the contents of the file. The default value of `model_ dir`is ``$TORCH_HOME/ models``where`` $TORCH_ Home``defaults to ``~ / .torch``. The default directory can be overriden with the ``$ TORCH_MODEL_ ZOO`` environment variable. Args: url (string): URL of the object to download model_dir (string, optional): directory in which to save the object map_location (optional): a function ora dict specifying how to remap storage locations (see torch.load) progress (bool Optional): whether or not to display a progress bar to stderr Example: > state_dict = torch.utils.model_zoo.load_url ('https://s3.amazonaws.com/pytorch/models/resnet18-5c106cde.pth') "if model_dir is None: torch_home = os.path.expanduser (os.getenv (' TORCH_HOME','~ / .torch')) model_dir = os.getenv ('TORCH_MODEL_ZOO', os.path.join (torch_home) 'models')) if not os.path.exists (model_dir): os.makedirs (model_dir) parts = urlparse (url) filename = os.path.basename (parts.path) cached_file = os.path.join (model_dir, filename) if not os.path.exists (cached_file): sys.stderr.write (' Downloading: "{}" to {}\ n'.format (url Cached_file) hash_prefix = HASH_REGEX.search (filename) .group (1) _ download_url_to_file (url, cached_file, hash_prefix, progress=progress) return torch.load (cached_file, map_location=map_location) are all the contents of the article "sample Analysis of torchvision.models in PyTorch" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.