The traditional model of neural network is called multilayer perceptrons. They are usually made up of a series of interconnected layers. The input layer is where the data enters the network, and the output layer is where the network delivers the output.
The input layer is usually connected to one or more hidden layers, which modify and process the data before it reaches the output layer. The hidden layers are what make neural networks so powerful: they can learn complicated functions that might be difficult for a programmer to specify in code.
In the previous tutorial, we built a neural network with only a couple of hidden neurons. Here, you will implement a neural network by adding more hidden neurons to it. This will estimate more complex function for us in order to fit the data.
During the implementation process, you’ll learn:
How to build a neural network with more hidden neurons in PyTorch.
How to estimate complex functions using neural networks by adding more hidden neurons to the network.
How to train a neural network in PyTorch.
Kick-start your project with my book Deep Learning with PyTorch. It provides self-study tutorials with working code.
Let’s get started.
Neural Network with More Hidden Neurons. Picture by Kdwk Leung. Some rights reserved.
Overview
This tutorial is in three parts; they are
Preparing the Data
Build the Model Architecture
Train the Model
Preparing the Data
Let’s build a Data class that extends the Dataset class from PyTorch. You use it to create a dataset of 100 synthetic values ranging from $-50$ to $50$. The x tensor stores the values in the specified range, while the y tensor is a corresponding tensor of zeros with the same shape as x.
Next, you use a for loop to set the values in x and y tensors based on the values in x. If a value in x is between $-20$ and $20$, the corresponding value in y is set to 1 and if a value in x is between $-30$ and $-20$ or between $20$ and $30$, the corresponding value in y is set to 0. Similarly, If a value in x is between $-40$ and $-30$ or between $30$ and $40$, the corresponding value in y is set to 1. Otherwise, the corresponding value in y is set to 0.
In the Data class, the __getitem__() method has been used to retrieve the x and y values at a specified index in the dataset. The __len__() method returns the length of the dataset. With these, you can obtain a sample from the dataset using data[i] and tell the size of the dataset using len(data). This class can be used to create a data object that can be passed to a PyTorch data loader to train a machine learning model.
Note that you are building this complex data object to see how well our neural network with more hidden neurons estimates the function. Here is how the code of the data object will look like.
If you run this function, you can see the data looks like the following:
1
plot_data(dataset.x,dataset.y,leg=False)
Want to Get Started With Deep Learning with PyTorch?
Take my free email crash course now (with sample code).
Click to sign-up and also get a free PDF Ebook version of the course.
Build the Model Architecture
Below, you will define a NeuralNetwork class to build a custom model architecture using nn.Module from PyTorch. This class represents a simple neural network with an input layer, a hidden layer, and an output layer.
The __init__() method is used to initialize the neural network by defining the layers in the network. The forward method is used to define the forward pass through the network. In this case, a sigmoid activation function is applied to the output of both input and output layers. This means that the output of the network will be a value between 0 and 1.
Finally, you will create an instance of the NeuralNetwork class and store it in the model variable. The model is initialized with an input layer having 1 input neuron, a hidden layer having 15 hidden neurons, and an output layer having 1 output neuron. This model is now ready to be trained on some data.
Let’s define the criterion, optimizer, and dataloader. You should use binary cross entropy loss as the dataset is a classification with two classes. Adam optimizer is used, with a batch size of 32. The learning rate is set to 0.01 which determines how model weights are updated during training. The loss function is used to evaluate the model performance, the optimizer updates the weights, and the data loader divides the data into batches for efficient processing.
Now, let’s build a training loop for 7000 epochs and visualize the results during training. You’ll see how well our model estimates the data points as the training progresses.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
n_epochs=7000# number of epochs to train the model
LOSS=[]# list to store the loss values after each epoch
# train the model for n_epochs
forepoch inrange(n_epochs):
total=0# variable to store the total loss for this epoch
# iterate over the data in the data loader
forx,yindata_loader:
# zero the gradients of the model
optimizer.zero_grad()
# make a prediction using the model
yhat=model(x)
# compute the loss between the predicted and true values
loss=criterion(yhat,y)
# compute the gradients of the model with respect to the loss
loss.backward()
# update the model parameters
optimizer.step()
# add the loss value to the total loss for this epoch
total+=loss.item()
# after each epoch, check if the epoch number is divisible by 200
ifepoch%1000==0:
# if it is, plot the current data and model using the PlotData function
# add the total loss for this epoch to the LOSS list
LOSS.append(total)
When you run this loop, you will see at the first epoch, the neural network modelled the dataset poorly, like the following:
But the accuracy improved as the training progressed. After the training loop completed, we can see the result as the neural network modelled the data like the following:
1
2
# plot after training loop ended
plot_data(dataset.x,dataset.y,model)
and the corresponding history of loss metric can be plot like the following:
1
2
3
4
5
6
7
# create a plot of the loss over epochs
plt.figure()
plt.plot(LOSS)
plt.xlabel('epochs')
plt.ylabel('loss')
# show the plot
plt.show()
As you can see, our model estimated the function quite well but not perfect. The input of range 20 to 40, for example, isn’t predicted right. You may try to expand the network to add one more layer, such as the following, and see if it will make any difference.
It provides self-study tutorials with hundreds of working code to turn you from a novice to expert. It equips you with tensor operation, training, evaluation, hyperparameter optimization,
and much more...
Kick-start your deep learning journey with hands-on exercises
How to choose the number of hidden layers?