Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "the method of conversion between torch.utils.data.DataLoader and iterator". The editor shows you the operation process through an actual case. The operation method is simple, fast and practical. I hope this article "the method of conversion between torch.utils.data.DataLoader and iterator" can help you solve the problem.

When doing experiments, we often use open source data sets for testing. There are many datasets built into Pytorch, which we often use the DataLoader class to load.

As in the following one, we use the DataLoader class to load the FashionMNIST dataset in torch.vision.

From torch.utils.data import DataLoaderfrom torchvision import datasetsfrom torchvision.transforms import ToTensorimport matplotlib.pyplot as plttraining_data = datasets.FashionMNIST (root= "data", train=True, download=True, transform=ToTensor ()) test_data = datasets.FashionMNIST (root= "data", train=False, download=True, transform=ToTensor ())

Let's next define the Dataloader object to load these two datasets:

Train_dataloader = DataLoader (training_data, batch_size=64, shuffle=True) test_dataloader = DataLoader (test_data, batch_size=64, shuffle=True)

So what kind of train_dataloader is this?

Print (type (train_dataloader)) #

We can convert it to an iterator type.

Print (type (iter (train_dataloader) #

Then use next (iter (train_dataloader)) to fetch data from the iterator, as follows:

Train_features, train_labels = next (iter (train_dataloader)) print (f "Feature batch shape: {train_features.size ()}") print (f "Labels batch shape: {train_labels.size ()}") img = train_features [0] .squeeze () label = train_labels [0] plt.imshow (img, cmap= "gray") plt.show () print (f "Label: {label}")

You can see that we have successfully obtained the information of the first image in the dataset, and the console prints:

Feature batch shape: torch.Size ([64,1,28,28]) Labels batch shape: torch.Size ([64]) Label: 2

The visual display of the picture is as follows:

However, some readers may wonder that most of the time we do not cast the DataLoader type into an iterator type, most of the time we will write the following code:

For train_features, train_labels in train_dataloader: print (train_features.shape) # torch.Size ([64,1,28,28]) print (train_features [0] .shape) # torch.Size ([1,28,28]) print (train_features [0] .squeeze (). Shape) # torch.Size ([28,28]) img = train_features [0] .squeeze () label = train_labels [0] plt.imshow (img Cmap= "gray") plt.show () print (f "Label: {label}")

As you can see, the code can also iterate the training data normally, and the console printout of the first three samples is as follows:

Torch.Size ([64,1,28,28]) torch.Size ([1,28,28]) torch.Size ([28,28]) Label: 7torch.Size ([64,1,28,28]) torch.Size ([1,28,28]) torch.Size ([28,28]) Label: 4torch.Size ([64,1,28,28]) torch.Size ([1,28,28]) torch.Size ([28,28]) Label: 1

So why don't we explicitly convert Dataloader to iterator types here? it's actually a mechanism for Python language for loops, once we use for. In... To iterate over an object, the Python interpreter will secretly and automatically create an iterator for us, that is to say

For train_features, train_labels in train_dataloader:

In fact, it is equivalent to

For train_features, train_labels in iter (train_dataloader):

Further, this is actually equivalent to

Train_iterator = iter (train_dataloader) try: while True: train_features, train_labels = next (train_iterator) except StopIteration: pass

By extension, when we iterate over the list directly with Python iterations:

For x in [1, 2, 3, 4]:

In fact, the Python interpreter has implicitly converted to an iterator for us:

List_iterator = iter ([1, 2, 3, 4]) try: while True: X = next (list_iterator) except StopIteration: pass's content on "how to convert torch.utils.data.DataLoader to iterators" ends here. Thank you for reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.