Shulou(Shulou.com)05/31 Report--

This article mainly introduces the relevant knowledge of "Pytorch deep learning classic convolution neural network resnet module case analysis". The editor shows you the operation process through the actual case, and the operation method is simple, fast and practical. I hope this article "Pytorch deep learning classic convolution neural network resnet module case analysis" can help you solve the problem.

Preface

With the continuous development of deep learning, the network structure has been continuously optimized from Alexnet to VGG, but in the process of VGG network research, people have found that with the continuous improvement of network depth, the accuracy has not been improved, as shown in the figure:

People feel that deep learning has stopped at this point, and can not continue to study, but after a period of development, residual network (resnet) has solved this problem.

1. Resnet

As shown in the figure: to put it simply, it means to retain the previous features. sometimes when the image is convoluted for feature extraction, the result is not as good as before, so resnet proposes to retain the previous features, which needs to be processed here, which will be described in detail in the following code explanation.

II. Resnet network structure

This article will mainly introduce resnet18

3. Resnet181. Guide package import torchimport torchvision.transforms as transimport torchvision as tvimport torch.nn as nnfrom torch.autograd import Variablefrom torch.utils import datafrom torch.optim import lr_scheduler2. Residual module

The function of this module is shown in the figure:

Class tiao (nn.Module): def _ init__ (self,shuru,shuchu): super (tiao, self). _ _ init__ () self.conv1=nn.Conv2d (in_channels=shuru,out_channels=shuchu,kernel_size= (3), padding= (1)) self.bath=nn.BatchNorm2d (shuchu) self.relu=nn.ReLU () def forward (self X): x1=self.conv1 (x) x2=self.bath (x1) x3=self.relu (x2) x4=self.conv1 (x3) x5=self.bath (x4) x6=self.relu (x5) x7=x6+x return x72. Double the number of channels residual module

The module completes the function as shown in the figure:

In this module, it should be noted that the number of channels in the original image should be doubled, otherwise it cannot be added later.

Class tiao2 (nn.Module): def _ init__ (self,shuru): super (tiao2, self). _ _ init__ () self.conv1=nn.Conv2d (in_channels=shuru,out_channels=shuru*2,kernel_size= (3), stride= (2), padding= (1)) self.conv11=nn.Conv2d (in_channels=shuru,out_channels=shuru*2,kernel_size= (1) Stride= (2) self.batch=nn.BatchNorm2d (shuru*2) self.relu=nn.ReLU () self.conv2=nn.Conv2d (in_channels=shuru*2,out_channels=shuru*2,kernel_size= (3), stride= (1), padding= (1)) def forward (self) X): x1=self.conv1 (x) x2=self.batch (x1) x3=self.relu (x2) x4=self.conv2 (x3) x5=self.batch (x4) x6=self.relu (x5) x11=self.conv11 (x) x7=x11+x6 return x73.rensnet18 module class resnet18 (nn.Module): def _ init__ (self): super (resnet18 Self). _ init__ () self.conv1=nn.Conv2d (in_channels=3,out_channels=64,kernel_size= (7), stride= (2) Padding= (3) self.bath=nn.BatchNorm2d (64) self.relu=nn.ReLU () self.max=nn.MaxPool2d (2) self.tiao1=tiao (64) self.tiao2=tiao (64) self.tiao3=tiao2 (64) self.tiao4=tiao (128128) self.tiao5=tiao2 (256256) self.tiao6=tiao (256256) self.tiao7=tiao2 (512512) Self.a=nn.AdaptiveAvgPool2d (output_size= (1)) self.l=nn.Linear (512) def forward (self) X): x1=self.conv1 (x) x2=self.bath (x1) x3=self.relu (x2) x4=self.tiao1 (x3) x5=self.tiao2 (x4) x6=self.tiao3 (x5) x7=self.tiao4 (x6) x8=self.tiao5 (x7) x9=self.tiao6 (x8) x10=self.tiao7 (x9) x11=self.tiao8 (x10) x12=self.a (x11) x13=x12.view (x12.size () [0] -1) x14=self.l (x13) return x14

To put it simply, this network has 16-layer convolution, 1-layer full connection, relatively few training parameters, and the model is relatively simple.

4. Data test model=resnet18 (). Cuda () input=torch.randn. Cuda () output=model (input) print (output)

5. Loss function, optimizer

Loss function

Loss=nn.CrossEntropyLoss ()

In the optimizer, the learning rate is automatically attenuated every 10 steps

Opt=torch.optim.SGD (model.parameters () lr=0.001,momentum=0.9) exp_lr=lr_scheduler.StepLR (opt,step_size=10,gamma=0.1) opt=torch.optim.SGD (model.parameters (), lr=0.001,momentum=0.9) exp_lr=lr_scheduler.StepLR (opt,step_size=10,gamma=0.1)

We can take a look at the comparison chart here and find that the loss decline rate will be faster with the addition of automatic attenuation of learning rate, which shows that the fitting effect of the model is better.

6. Loading datasets, data enhancement

Here we still choose the cifar10 dataset. First, we enhance the data to increase the pan-Chinese ability of the model.

Transs=trans.Compose ([trans.Resize (256), trans.RandomHorizontalFlip (), trans.RandomCrop (64), trans.ColorJitter (brightness=0.5,contrast=0.5,hue=0.3), trans.ToTensor (), trans.Normalize)

Brightness (luminance) contrast (contrast) saturation (saturation) hue (hue) in the ColorJitter function

Load the cifar10 dataset:

Train=tv.datasets.CIFAR10 (root=r'E:\ desktop\ material\ cv3\ dataset\ cifar-10-batches-py', train=True, download=True, transform=transs) trainloader=data.DataLoader (train, num_workers=4, batch_size=8, shuffle=True, drop_last=True) 7. Training data for i in range (3): running_loss=0 for index,data in enumerate (trainloader): X training data x=x.cuda () y=y.cuda () x=Variable (x) y=Variable (y) opt.zero_grad () h=model (x) loss1=loss (h Y) loss1.backward () opt.step () running_loss+=loss1.item () if index0==99: avg_loos=running_loss/100 running_loss=0 print ("avg_loss", avg_loos) 8. Save the model torch.save (model.state_dict (), 'resnet18.pth') 9. Load test set data for model testing

First load the trained model

Model.load_state_dict (torch.load ('resnet18.pth'), False)

Read data

Test = tv.datasets.ImageFolder (root=r'E:\ desktop\ material\ cv3\ data', transform=transs,) testloader = data.DataLoader (test, batch_size=16, shuffle=False,)

test data

Acc=0total=0 for data in testloader: inputs,indel=data out=model (inputs.cuda ()) _, prediction=torch.max (out.cpu (), 1) total+=indel.size (0) b = (prediction==indel) acc+=b.sum () print ("accuracy% d%"% (100*acc/total)) 4. Deep comparison of resnet

It is mentioned above that the deeper the VGG network is, the lower the accuracy is. In order to solve this problem, the residual network (resnet) is proposed, so whether this problem will occur in the resnet network in the end.

As shown in the figure: with the continuous improvement of the training level, the model is getting better and better, and the problem of VGG network has been solved successfully. Up to now, the residual network is still used by most people.

This is the end of the introduction of "Pytorch deep learning classic convolution neural network resnet module example analysis". Thank you for your reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.