Building an Image Classifier with a Single-Layer Neural Network in PyTorch

By Muhammad Asad Iqbal Khan on April 8, 2023 in Deep Learning with PyTorch 12

A single-layer neural network, also known as a single-layer perceptron, is the simplest type of neural network. It consists of only one layer of neurons, which are connected to the input layer and the output layer. In case of an image classifier, the input layer would be an image and the output layer would be a class label.

To build an image classifier using a single-layer neural network in PyTorch, you’ll first need to prepare your data. This typically involves loading the images and labels into a PyTorch dataloader, and then splitting the data into training and validation sets. Once your data is prepared, you can define your neural network.

Next, you can use PyTorch’s built-in functions to train the network on your training data and evaluate its performance on your validation data. You’ll also need to pick an optimizer such as stochastic gradient descent (SGD) and a loss function like cross-entropy loss.

Note that a single layer neural network might not be ideal for every task, but it can be good as simple classifier and also can be helpful for you to understand the inner workings of the neural network and to be able to debug it.

So, let’s build our image classifier. In the process you’ll learn:

How to use and preprocess built-in datasets in PyTorch.
How to build and train custom neural networks in PyTorch.
How to build a step-by-step image classifier in PyTorch.
How to make predictions using the trained model in PyTorch.

Let’s get started.

Building an Image Classifier with a Single-Layer Neural Network in PyTorch.
Picture by Alex Fung. Some rights reserved.

Overview

This tutorial is in three parts; they are

Preparing the Dataset
Build the Model Architecture
Train the Model

Preparing the Dataset

In this tutorial, you will use the CIFAR-10 dataset. It is a dataset for image classification, consisting of 60,000 color images of 32×32 pixels in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images. The classes include airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. CIFAR-10 is a popular dataset for machine learning and computer vision research, as it is relatively small and simple, yet challenging enough to require the use of deep learning methods. This dataset can be easily imported into PyTorch library.

Here is how you do that.

import torch
import torchvision
import torchvision.transforms as transforms

# import the CIFAR-10 dataset
train_set = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transforms.ToTensor())
test_set = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transforms.ToTensor())

import torch

import torchvision

import torchvision.transforms as transforms

# import the CIFAR-10 dataset

train_set = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transforms.ToTensor())

test_set = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transforms.ToTensor())

If you never downloaded the dataset before, you may see this code show you where the images are downloaded from:

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz
  0%|          | 0/170498071 [00:00<!--?, ?it/s]
Extracting ./data/cifar-10-python.tar.gz to ./data
Files already downloaded and verified

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz

0%| | 0/170498071 [00:00<!--?, ?it/s]

Extracting ./data/cifar-10-python.tar.gz to ./data

Files already downloaded and verified

You specified the root directory where the dataset should be downloaded, and setting train=True to import the training set, and train=False to import the test set. The download=True argument will download the dataset if it’s not already present in the specified root directory.

Building the Neural Network Model

Let’s define a simple neural network SimpleNet that inherits from torch.nn.Module. The network has two fully connected (fc) layers, fc1 and fc2, defined in the __init__ method. The first fully connected layer fc1 takes in the image as input and has 100 hidden neurons. Similarly, the second fully connected layer fc2 has 100 input neurons and num_classes output neurons. The num_classes parameter defaults to 10 as there are 10 classes.

Moreover, the forward method defines the forward pass of the network, where the input x is passed through the layers defined in the __init__ method. The method first reshapes the input tensor x to have a desired shape using the view method. The input then passes through the fully connected layers along with their activation functions and, finally, an output tensor is returned.

Kick-start your project with my book Deep Learning with PyTorch. It provides self-study tutorials with working code.

Here is the code for all explained above.

# Create the Data object
dataset = Data()

1 2	# Create the Data object dataset = Data()

And, write a function to visualize this data, which will also be useful when you train the model later.

import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self, num_classes=10):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(32*32*3, 100) # Fully connected layer with 100 hidden neurons
        self.fc2 = nn.Linear(100, num_classes) # Fully connected layer with num_classes outputs

    def forward(self, x):
        x = x.view(-1, 32*32*3) # reshape the input tensor
        x = self.fc1(x)
        x = torch.relu(x)
        x = self.fc2(x)
        return x

import torch.nn as nn

class SimpleNet(nn.Module):

def __init__(self, num_classes=10):

super(SimpleNet, self).__init__()

self.fc1 = nn.Linear(32*32*3, 100) # Fully connected layer with 100 hidden neurons

self.fc2 = nn.Linear(100, num_classes) # Fully connected layer with num_classes outputs

def forward(self, x):

x = x.view(-1, 32*32*3) # reshape the input tensor

x = self.fc1(x)

x = torch.relu(x)

x = self.fc2(x)

return x

Now, let’s instantiate the model object.

# Instantiate the model
model = SimpleNet()

1 2	# Instantiate the model model = SimpleNet()

Want to Get Started With Deep Learning with PyTorch?

Take my free email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Training the Model

You will create two instances of PyTorch’s DataLoader class, for training and testing respectively. In train_loader, you set the batch size at 64 and shuffle the training data randomly by setting shuffle=True.

Then, you will define the functions for cross entropy loss and Adam optimizer for training the model. You set the learning rate at 0.001 for the optimizer.

It is similar for test_loader, except we don’t need to shuffle.

# Load the data into PyTorch DataLoader
train_loader = torch.utils.data.DataLoader(train_set, batch_size=64, shuffle=True)
test_loader = torch.utils.data.DataLoader(test_set, batch_size=64, shuffle=False)

# Define the loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

# Load the data into PyTorch DataLoader

train_loader = torch.utils.data.DataLoader(train_set, batch_size=64, shuffle=True)

test_loader = torch.utils.data.DataLoader(test_set, batch_size=64, shuffle=False)

# Define the loss function and optimizer

criterion = nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

Finally, let’s set a training loop to train our model for a few epochs. You will define some empty lists to store the values of the loss and accuracy metrices for loss and accuracy.

# train the model
num_epochs = 20
train_loss_history = []
train_acc_history = []
val_loss_history = []
val_acc_history = []

# Loop through the number of epochs
for epoch in range(num_epochs):
    train_loss = 0.0
    train_acc = 0.0
    val_loss = 0.0
    val_acc = 0.0

    # set model to train mode
    model.train()
    # iterate over the training data
    for inputs, labels in train_loader:
        optimizer.zero_grad()
        outputs = model(inputs)
        #compute the loss
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        # increment the running loss and accuracy
        train_loss += loss.item()
        train_acc += (outputs.argmax(1) == labels).sum().item()

    # calculate the average training loss and accuracy
    train_loss /= len(train_loader)
    train_loss_history.append(train_loss)
    train_acc /= len(train_loader.dataset)
    train_acc_history.append(train_acc)

    # set the model to evaluation mode
    model.eval()
    with torch.no_grad():
        for inputs, labels in test_loader:
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            val_loss += loss.item()
            val_acc += (outputs.argmax(1) == labels).sum().item()

    # calculate the average validation loss and accuracy
    val_loss /= len(test_loader)
    val_loss_history.append(val_loss)
    val_acc /= len(test_loader.dataset)
    val_acc_history.append(val_acc)

    print(f'Epoch {epoch+1}/{num_epochs}, train loss: {train_loss:.4f}, train acc: {train_acc:.4f}, val loss: {val_loss:.4f}, val acc: {val_acc:.4f}')

# train the model

num_epochs = 20

train_loss_history = []

train_acc_history = []

val_loss_history = []

val_acc_history = []

# Loop through the number of epochs

for epoch in range(num_epochs):

train_loss = 0.0

train_acc = 0.0

val_loss = 0.0

val_acc = 0.0

# set model to train mode

model.train()

# iterate over the training data

for inputs, labels in train_loader:

optimizer.zero_grad()

outputs = model(inputs)

#compute the loss

loss = criterion(outputs, labels)

loss.backward()

optimizer.step()

# increment the running loss and accuracy

train_loss += loss.item()

train_acc += (outputs.argmax(1) == labels).sum().item()

# calculate the average training loss and accuracy

train_loss /= len(train_loader)

train_loss_history.append(train_loss)

train_acc /= len(train_loader.dataset)

train_acc_history.append(train_acc)

# set the model to evaluation mode

model.eval()

with torch.no_grad():

for inputs, labels in test_loader:

outputs = model(inputs)

loss = criterion(outputs, labels)

val_loss += loss.item()

val_acc += (outputs.argmax(1) == labels).sum().item()

# calculate the average validation loss and accuracy

val_loss /= len(test_loader)

val_loss_history.append(val_loss)

val_acc /= len(test_loader.dataset)

val_acc_history.append(val_acc)

print(f'Epoch {epoch+1}/{num_epochs}, train loss: {train_loss:.4f}, train acc: {train_acc:.4f}, val loss: {val_loss:.4f}, val acc: {val_acc:.4f}')

Running this loop will print you the following:

Epoch 1/20, train loss: 1.8757, train acc: 0.3292, val loss: 1.7515, val acc: 0.3807
Epoch 2/20, train loss: 1.7254, train acc: 0.3862, val loss: 1.6850, val acc: 0.4008
Epoch 3/20, train loss: 1.6548, train acc: 0.4124, val loss: 1.6692, val acc: 0.3987
Epoch 4/20, train loss: 1.6150, train acc: 0.4268, val loss: 1.6052, val acc: 0.4265
Epoch 5/20, train loss: 1.5874, train acc: 0.4343, val loss: 1.5803, val acc: 0.4384
Epoch 6/20, train loss: 1.5598, train acc: 0.4424, val loss: 1.5928, val acc: 0.4315
Epoch 7/20, train loss: 1.5424, train acc: 0.4506, val loss: 1.5489, val acc: 0.4514
Epoch 8/20, train loss: 1.5310, train acc: 0.4568, val loss: 1.5566, val acc: 0.4454
Epoch 9/20, train loss: 1.5116, train acc: 0.4626, val loss: 1.5501, val acc: 0.4442
Epoch 10/20, train loss: 1.5005, train acc: 0.4677, val loss: 1.5282, val acc: 0.4598
Epoch 11/20, train loss: 1.4911, train acc: 0.4702, val loss: 1.5310, val acc: 0.4629
Epoch 12/20, train loss: 1.4804, train acc: 0.4756, val loss: 1.5555, val acc: 0.4457
Epoch 13/20, train loss: 1.4743, train acc: 0.4762, val loss: 1.5207, val acc: 0.4629
Epoch 14/20, train loss: 1.4658, train acc: 0.4792, val loss: 1.5177, val acc: 0.4570
Epoch 15/20, train loss: 1.4608, train acc: 0.4819, val loss: 1.5529, val acc: 0.4527
Epoch 16/20, train loss: 1.4539, train acc: 0.4832, val loss: 1.5066, val acc: 0.4645
Epoch 17/20, train loss: 1.4486, train acc: 0.4863, val loss: 1.4874, val acc: 0.4727
Epoch 18/20, train loss: 1.4503, train acc: 0.4866, val loss: 1.5318, val acc: 0.4575
Epoch 19/20, train loss: 1.4383, train acc: 0.4910, val loss: 1.5065, val acc: 0.4673
Epoch 20/20, train loss: 1.4348, train acc: 0.4897, val loss: 1.5127, val acc: 0.4679

Epoch 1/20, train loss: 1.8757, train acc: 0.3292, val loss: 1.7515, val acc: 0.3807

Epoch 2/20, train loss: 1.7254, train acc: 0.3862, val loss: 1.6850, val acc: 0.4008

Epoch 3/20, train loss: 1.6548, train acc: 0.4124, val loss: 1.6692, val acc: 0.3987

Epoch 4/20, train loss: 1.6150, train acc: 0.4268, val loss: 1.6052, val acc: 0.4265

Epoch 5/20, train loss: 1.5874, train acc: 0.4343, val loss: 1.5803, val acc: 0.4384

Epoch 6/20, train loss: 1.5598, train acc: 0.4424, val loss: 1.5928, val acc: 0.4315

Epoch 7/20, train loss: 1.5424, train acc: 0.4506, val loss: 1.5489, val acc: 0.4514

Epoch 8/20, train loss: 1.5310, train acc: 0.4568, val loss: 1.5566, val acc: 0.4454

Epoch 9/20, train loss: 1.5116, train acc: 0.4626, val loss: 1.5501, val acc: 0.4442

Epoch 10/20, train loss: 1.5005, train acc: 0.4677, val loss: 1.5282, val acc: 0.4598

Epoch 11/20, train loss: 1.4911, train acc: 0.4702, val loss: 1.5310, val acc: 0.4629

Epoch 12/20, train loss: 1.4804, train acc: 0.4756, val loss: 1.5555, val acc: 0.4457

Epoch 13/20, train loss: 1.4743, train acc: 0.4762, val loss: 1.5207, val acc: 0.4629

Epoch 14/20, train loss: 1.4658, train acc: 0.4792, val loss: 1.5177, val acc: 0.4570

Epoch 15/20, train loss: 1.4608, train acc: 0.4819, val loss: 1.5529, val acc: 0.4527

Epoch 16/20, train loss: 1.4539, train acc: 0.4832, val loss: 1.5066, val acc: 0.4645

Epoch 17/20, train loss: 1.4486, train acc: 0.4863, val loss: 1.4874, val acc: 0.4727

Epoch 18/20, train loss: 1.4503, train acc: 0.4866, val loss: 1.5318, val acc: 0.4575

Epoch 19/20, train loss: 1.4383, train acc: 0.4910, val loss: 1.5065, val acc: 0.4673

Epoch 20/20, train loss: 1.4348, train acc: 0.4897, val loss: 1.5127, val acc: 0.4679

As you can see, the single-layer classifier is trained for only 20 epochs and it achieved a validation accuracy of around 47 percent. Train it for more epochs and you may get a decent accuracy. Similarly, our model had only a single layer with 100 hidden neurons. If you add some more layers, the accuracy may significantly improve.

Now, let’s plot loss and accuracy matrices to see how they look like.

import matplotlib.pyplot as plt

# Plot the training and validation loss
plt.plot(train_loss_history, label='train loss')
plt.plot(val_loss_history, label='val loss')
plt.legend()
plt.show()

# Plot the training and validation accuracy
plt.plot(train_acc_history, label='train acc')
plt.plot(val_acc_history, label='val acc')
plt.legend()
plt.show()

import matplotlib.pyplot as plt

# Plot the training and validation loss

plt.plot(train_loss_history, label='train loss')

plt.plot(val_loss_history, label='val loss')

plt.legend()

plt.show()

# Plot the training and validation accuracy

plt.plot(train_acc_history, label='train acc')

plt.plot(val_acc_history, label='val acc')

plt.legend()

plt.show()

The loss plot is like:And the accuracy plot is the following:

Here is how you can see how the model make predictions against the true labels.

import numpy as np

# get some validation data
for inputs, labels in test_loader:
    break  # this line stops the loop after the first iteration

# make predictions
outputs = model(inputs)
_, predicted = torch.max(outputs, 1)

# display the images and their labels
img_grid = torchvision.utils.make_grid(inputs)
img_grid = img_grid / 2 + 0.5     # unnormalize
npimg = img_grid.numpy()
plt.imshow(np.transpose(npimg, (1, 2, 0)))

print('True Labels: ', labels)
print('Predicted Labels: ', predicted)

import numpy as np

# get some validation data

for inputs, labels in test_loader:

break # this line stops the loop after the first iteration

# make predictions

outputs = model(inputs)

_, predicted = torch.max(outputs, 1)

# display the images and their labels

img_grid = torchvision.utils.make_grid(inputs)

img_grid = img_grid / 2 + 0.5 # unnormalize

npimg = img_grid.numpy()

plt.imshow(np.transpose(npimg, (1, 2, 0)))

print('True Labels: ', labels)

print('Predicted Labels: ', predicted)

The printed labels are as following:

True Labels:  tensor([3, 8, 8, 0, 6, 6, 1, 6, 3, 1, 0, 9, 5, 7, 9, 8, 5, 7, 8, 6, 7, 0, 4, 9,
        5, 2, 4, 0, 9, 6, 6, 5, 4, 5, 9, 2, 4, 1, 9, 5, 4, 6, 5, 6, 0, 9, 3, 9,
        7, 6, 9, 8, 0, 3, 8, 8, 7, 7, 4, 6, 7, 3, 6, 3])
Predicted Labels:  tensor([3, 9, 8, 8, 4, 6, 3, 6, 2, 1, 8, 9, 6, 7, 1, 8, 5, 3, 8, 6, 9, 2, 0, 9,
        4, 6, 6, 2, 9, 6, 6, 4, 3, 3, 9, 1, 6, 9, 9, 5, 0, 6, 7, 6, 0, 9, 3, 8,
        4, 6, 9, 4, 6, 3, 8, 8, 5, 8, 8, 2, 7, 3, 6, 9])

True Labels: tensor([3, 8, 8, 0, 6, 6, 1, 6, 3, 1, 0, 9, 5, 7, 9, 8, 5, 7, 8, 6, 7, 0, 4, 9,

5, 2, 4, 0, 9, 6, 6, 5, 4, 5, 9, 2, 4, 1, 9, 5, 4, 6, 5, 6, 0, 9, 3, 9,

7, 6, 9, 8, 0, 3, 8, 8, 7, 7, 4, 6, 7, 3, 6, 3])

Predicted Labels: tensor([3, 9, 8, 8, 4, 6, 3, 6, 2, 1, 8, 9, 6, 7, 1, 8, 5, 3, 8, 6, 9, 2, 0, 9,

4, 6, 6, 2, 9, 6, 6, 4, 3, 3, 9, 1, 6, 9, 9, 5, 0, 6, 7, 6, 0, 9, 3, 8,

4, 6, 9, 4, 6, 3, 8, 8, 5, 8, 8, 2, 7, 3, 6, 9])

These labels are to correspond to the following images:

Summary

In this tutorial, you learned how you can build an image classifier using only a single-layer neural network. Particularly, you learned:

How to use and preprocess built-in datasets in PyTorch.
How to build and train custom neural networks in PyTorch.
How to build a step-by-step image classifier in PyTorch.
How to make predictions using the trained model in PyTorch.

12 Responses to Building an Image Classifier with a Single-Layer Neural Network in PyTorch

Tony the Riger January 20, 2023 at 8:35 am #

Where is Jason, please?

Reply
Chuck February 2, 2023 at 8:08 am #

Model worked but the codes after this i.e.,
plottiungs for accuracy/loss and prediction image,
the kernal died with this message –
‘The kernel appears to have died. It will restart automatically’. I tried several times but the result were same. Any suggestion for solving this? Thanmks.

Reply
- James Carmichael February 2, 2023 at 8:42 am #
  
  Hi Chuck…While we have not experienced this issue, the following resource may be helpful:
  
  https://stackoverflow.com/questions/47022997/jupyter-the-kernel-appears-to-have-died-it-will-restart-automatically
  
  Reply
Leo February 24, 2023 at 8:23 am #

# Create the Data object
“dataset = Data()”

Pytorch reported Error after this. The Data Class was not created?

Reply
- James Carmichael February 24, 2023 at 11:03 am #
  
  Hi Leo…Please elaborate on your question so that we may better assist you. That is…what errors are you receiving?
  
  Reply
  - Fabio April 20, 2023 at 8:16 pm #
    
    in this tutorial it must have been missed, but it was present in other tutorials
    
    use this before dataset = Data():
    
    # Creating the dataset class
    class Data(Dataset):
    def __init__(self):
    self.x = torch.arange(-2, 2, 0.1).view(-1, 1)
    self.y = torch.zeros(self.x.shape[0], 1)
    self.y[self.x[:, 0] > 0.2] = 1
    self.len = self.x.shape[0]
    
    def __getitem__(self, idx):
    return self.x[idx], self.y[idx]
    
    def __len__(self):
    return self.len
    
    Reply
    - James Carmichael April 21, 2023 at 9:29 am #
      
      Thank you Fabio for your feedback and suggestion!
      
      Reply
      - Narae July 31, 2023 at 5:22 pm #
        
        Hi,
        
        I think the
        
        dataset = Data()
        
        part was left in the code by mistake? I doesn’t seemed to be used anywhere in the code below it, and the Data() method is never defined prior to its use.
      - James Carmichael August 1, 2023 at 9:15 am #
        
        Hi Narae…Thanks for the feedback!
Leo February 24, 2023 at 11:37 am #

Hi Kames,
Thank yuo for replying

The code and error are here:

————————————————-

import torch
import torchvision
import torchvision.transforms as transforms

# import the CIFAR-10 dataset
train_set = torchvision.datasets.CIFAR10(root=’./data’, train=True, download=True, transform=transforms.ToTensor())
test_set = torchvision.datasets.CIFAR10(root=’./data’, train=False, download=True, transform=transforms.ToTensor())

# Create the Data object
dataset = Data()

————————————————————–

NameError Traceback (most recent call last)
Input In [1], in ()
7 test_set = torchvision.datasets.CIFAR10(root=’./data’, train=False, download=True, transform=transforms.ToTensor())
9 # Create the Data object
—> 10 dataset = Data()

NameError: name ‘Data’ is not defined

Reply
Sean O'Connor April 1, 2023 at 3:46 pm #

Even single neural network layers don’t scale well with width.
If the width is n then the number of multiply add operations required is n squared.
What starts off as something reasonable at say n=8 giving 64 multiply adds, starts to get unreasonable at n=256 giving 65536 multiply adds.
However by using a combining algorithm you can restrain the costs:
https://ai462qqq.blogspot.com/2023/03/switch-net-4-reducing-cost-of-neural.html

Reply
- James Carmichael April 2, 2023 at 6:21 am #
  
  Thank you for your feedback and contribution Sean!
  
  Reply

Navigation

Building an Image Classifier with a Single-Layer Neural Network in PyTorch

Overview

Preparing the Dataset

Building the Neural Network Model

Want to Get Started With Deep Learning with PyTorch?

Training the Model

Summary

Get Started on Deep Learning with PyTorch!

Learn how to build deep learning models

Kick-start your deep learning journey with hands-on exercises

More On This Topic

12 Responses to Building an Image Classifier with a Single-Layer Neural Network in PyTorch

Leave a Reply Click here to cancel reply.