Shulou(Shulou.com)06/03 Report--

This article mainly shows you "pyTorch how to achieve softmax", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "pyTorch how to achieve softmax" this article.

Using PyTorch to implement linear model simulation dataset num_inputs = 2 # feature numbernum_examples = 1000 # number of training samples true_w = torch.tensor ([[2], [- 1000]) # True weight value true_b = torch.tensor (4.2) # Real biassamples = torch.normal (0Magne 1, (num_examples,num_inputs)) noise = torch.normal (0MagneOl, (num_examples) 1) labels = samples.matmul (true_w) + true_b + noise definition model class LinearNet (nn.Module): def _ init__ (self,in_features): super (). _ init__ () self.fc = nn.Linear (in_features=2,out_features=1) def forward (self) T): t = self.fc (t) return t load dataset import torch.utils.data as Datadataset = Data.TensorDataset (samples,labels) # similar to zip Package two tensors data_loader = Data.DataLoader (dataset,batch_size=100,shuffle=True) optimizernetwork = LinearNet (2) optimizer = optim.SGD (network.paramters (), lr=0.05) Model training for epoch in range (10): total_loss = 0 for data,label in data_loader: predict = network (data) loss = F.mse_loss (predict) Label) total_loss + = loss.item () optimizer.zero_grad () loss.backward () optimizer.step () print ('epoch',epoch,' loss',total_loss, 'weight',network.weight,' bias',network.bias) softmax regression model

Sotfmax is mainly used for classification tasks. What regression ends up with is a scalar and an output based on the linear addition of features in input. The result of the classification task is a category, which is discrete.

Suppose there is a batch of pictures that are 2 by 2 grayscale images, so that every two pixels in the picture is represented by a scalar. In these pictures, there are three kinds of small animals, the first is puppies, the second is kittens, and the third is rabbits.

Each picture has a total of four pixels, which can be regarded as four feature, assuming that the pictures of these three types of animals are linearly separable, and each category corresponds to a set of weight and a bias.

Which category can be determined according to the larger output value, but there is a problem, first of all, the output value has no clear meaning, and may be a range of real numbers. Secondly, it is difficult to measure the gap between the output value and the real value. Therefore, the softmax operation is used to convert the three output values into probability values, so that the output results satisfy the probability distribution. Label is encoded by one-hot, which is equivalent to the probability of the corresponding category is 1, so you can use cross_entropy to calculate loss.

Fashion-MNIST

The learning softmax model adopts Fashion-MNIST in torchvision.datasets.

Import torchvisionimport torchvision.transforms as transformstrain_set = torchvision.datasets.FashionMNIST (root='./data', train=True, download=True, transform=transforms.ToTensor ())

Transforms.ToTensor () converts a PIL picture with size (H x W x C) and data in (0255) or NumPy array with data type np.uint8 to Tensor with size C x H x W and data type torch.float32 and located at (0.01.0)

Len (train_set), len (test_set) > (60000, 100000)

Show the pictures in the dataset

Import matplotlib.pyplot as pltplt.figure (figsize= (10 off' 10)) for I, (image,lable) in enumerate (train_set,start=1): plt.subplot (1Magi 10 Magi I) plt.imshow (image.squeeze ()) plt.title (train_ set.classes [able]) plt.axis ('off') if I = 10: breakplt.show ()

Train_loader = torch.utils.data.DataLoader (train_set,batch_size=100,shuffle=True,num_workers=4) test_loader = torch.utils.data.DataLoader (test_set,batch_size=100,shuffle=False,num_workers=1) cross_entropydef net (samples,w,b): samples = samples.flatten (start_dim=1) # expand the three axes of cjournal into one feature axis The length is 28 * 28 samples = torch.exp (samples) # all elements are normalized by the index partial_sum = samples.sum (dim=1,keepdim=True) samples = samples / partial_sum # based on e, and the broadcast mechanism return samples.matmul (w) + b is also applied here.

I represents the type corresponding to label, pi is the prediction probability of the real kind, and log is the logarithm based on e.

Here, the function of the gather function is to get the probability value of the corresponding label on the predict. Note that the negative sign cannot be lost. The cross_entropy in the pytorch performs a softmax operation on the input first to ensure that the input is positive.

The realization of the model def net (samples,w,b): samples = samples.flatten (start_dim=1) # expands the three axes of the crecinct into a feature axis with a length of 28 * 28 samples = torch.exp (samples) # all elements are normalized with e-based index partial_sum = samples.sum (dim=1,keepdim=True) samples = samples / partial_sum # to obtain the probability Here we also apply the broadcast mechanism return samples.matmul (w) + b to easily implement softmaximport torchimport torchvisionimport torch.nn as nnimport torch.nn.functional as Fimport torch.utils.data as Dataimport torchvision.transforms as transformsimport torch.optim as optimimport torch.nn.init as initclass SoftmaxNet (nn.Module): def _ init__ (self,in_features,out_features): super (). _ init__ () self.fc = nn.Linear (in_features=in_features) by using PyTorch Out_features=out_features) def forward (self,t): t = t.flatten (start_dim=1) t = self.fc (t) return ttrain_set = torchvision.datasets.FashionMNIST (root='E:\ project\ python\ jupyterbook\ data', train=True, download=True, transform=transforms.ToTensor ()) test_set = torchvision.datasets.FashionMNIST (root='E:\ project\ python\ jupyterbook\ data', train=False, download=True Transform=transforms.ToTensor () train_loader = Data.DataLoader (train_set, batch_size=100, shuffle=True, # num_workers=2) test_loader = Data.DataLoader (test_set, batch_size=100, shuffle=False, # num_workers=2) @ torch.no_grad () def get_correct_nums (predict,labels): return predict.argmax (dim=1) .eq (labels). Sum (). Item () @ torch.no_grad () def evaluate (test_loader,net) Total_num): correct = 0 for image,label in test_loader: predict = net (image) correct + = get_correct_nums (predict,label) pass return correct / total_numnetwork = SoftmaxNet () optimizer = optim.SGD (network.parameters (), lr=0.05) for epoch in range (10): total_loss = 0 total_correct = 0 for image Label in train_loader: predict = network (image) loss = F.cross_entropy (predict,label) total_loss + = loss.item () total_correct + = get_correct_nums (predict,label) optimizer.zero_grad () loss.backward () optimizer.step () pass print ('epoch',epoch,' loss',total_loss, 'train_acc' Total_correct / len (train_set), 'test_acc',evaluate (test_loader,network,len (test_set) these are all the contents of the article "how pyTorch implements softmax" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.