Last Updated on December 19, 2022

Logistic regression is a statistical technique for modeling the probability of an event. It is often used in machine learning for making predictions. We apply logistic regression when a categorical outcome needs to be predicted.

In PyTorch, the construction of logistic regression is similar to that of linear regression. They both applied to linear inputs. But logistic regression is specifically classification problems, such as classifying into one of the two outcomes (0 or 1).

In this tutorial, we’ll focus on making predictions with logistic regression. We’ll learn how some of the useful packages in the PyTorch library can help easily create a logistic regression model. Particularly, we’ll learn:

- How to make predictions with logistic regression in PyTorch.
- The logistic function and its implementation on tensors.
- How to build a logistic regression model with
`nn.Sequential`

. - How to build a custom module for logistic regression.

Let’s get started.

## Overview

This tutorial is in four parts; they are

- Create Data Class
- Build the Model with
`nn.Module`

- Train with Mini-Batch Gradient Descent
- Plot the Progress

## What is a Logistic Function?

When class of a certain point in a dataset is calculated using a linear function, we get a positive or a negative number such as $-3$, $2$, $4$, etc. When we build a classifier, or specifically a binary classifier, we wish it can return either 0 or 1. A sigmoid or logistic function can be used in this case as this function always return a value between 0 and 1. Usually we will set a threshold, such as 0.5, to round up or round down the result to designate the output to one class or another.

In PyTorch, the logistic function is implemented by the `nn.Sigmoid()`

method. Let’s define a tensor by using the `range()`

method in PyTorch and apply the logistic function to observe the output.

1 2 3 4 5 6 |
import torch torch.manual_seed(42) xrange = torch.range(-50, 50, 0.5) sig_func = torch.nn.Sigmoid() y_pred = sig_func(xrange) |

Let’s see how the plot looks like.

1 2 3 4 5 6 |
import matplotlib.pyplot as plt plt.plot(xrange.numpy(), y_pred.numpy()) plt.xlabel('range') plt.ylabel('y_pred') plt.show() |

As you can see in the plot, the values of a logistic function range between 0 and 1, with the transition happen mostly around 0.

## Logistic Regression Model via `nn.Sequential`

The `nn.Sequential`

package in PyTorch enables us to build logistic regression model just like we can build our linear regression models. We simply need to define a tensor for input and process it through the model.

Let’s define a Logistic Regression model object that takes one-dimensional tensor as input.

1 2 |
... log_regr = torch.nn.Sequential(torch.nn.Linear(1, 1), torch.nn.Sigmoid()) |

This model has a linear function layer. The output from the linear function is passed on to the logistic function that makes the prediction.

We can check the list of model parameters using `parameters()`

method. The parameters should be randomly initialized in this case but we can see the shape match what we specified in the model above.

1 2 |
... print(list(log_regr.parameters())) |

Here’s what the output looks like.

1 2 3 |
[Parameter containing: tensor([[0.7645]], requires_grad=True), Parameter containing: tensor([0.8300], requires_grad=True)] |

Now, let’s define a one-dimensional tensor `x`

and make predictions with our logistic regression model.

1 |
x = torch.tensor([[1], [2], [3], [4]], dtype=torch.float32) |

We force the tensor to be in `float32`

type because this is what our model expects. Feeding this samples of data into the model, we will get the following predictions.

1 2 |
y_pred = log_regr(x) print("here is model prediction: ", y_pred) |

Its output is like the following:

1 2 3 4 |
here is model prediction: tensor([[0.8313], [0.9137], [0.9579], [0.9799]], grad_fn=<SigmoidBackward0>) |

Putting everything together, the following is the complete code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
import matplotlib.pyplot as plt import torch torch.manual_seed(42) xrange = torch.range(-50, 50, 0.5) sig_func = torch.nn.Sigmoid() y_pred = sig_func(xrange) plt.plot(xrange.numpy(), y_pred.numpy()) plt.xlabel('range') plt.ylabel('y_pred') plt.show() log_regr = torch.nn.Sequential(torch.nn.Linear(1, 1), torch.nn.Sigmoid()) print(list(log_regr.parameters())) x = torch.tensor([[1], [2], [3], [4]], dtype=torch.float32) y_pred = log_regr(x) print("here is model prediction: ", y_pred) |

## Custom Module for Logistic Regression

Knowing how to build custom modules is necessary when you work on advanced deep learning solutions. We can try out the syntax and build our custom logistic regerssion module. This should work identically to the `nn.Sequential`

model above.

We’ll define the class and inherit all the methods and attributes from the `nn.Module`

package. In the `forward()`

function of the class, we’ll use `sigmoid()`

method which takes the output from the linear function of the class and makes the prediction.

1 2 3 4 5 6 7 8 9 10 11 |
# build custom module for logistic regression class LogisticRegression(torch.nn.Module): # build the constructor def __init__(self, n_inputs): super(LogisticRegression, self).__init__() self.linear = torch.nn.Linear(n_inputs, 1) # make predictions def forward(self, x): y_pred = torch.sigmoid(self.linear(x)) return y_pred |

We can instantiate the class object.

1 2 |
... log_regr_cus = LogisticRegression(1) |

Now, let’s make predictions for the tensor `x`

we defined above.

1 2 3 |
... y_pred = log_regr_cus(x) print("here is model prediction: ", y_pred) |

The output would be:

1 2 3 4 |
here is model prediction: tensor([[0.6647], [0.6107], [0.5537], [0.4954]], grad_fn=<SigmoidBackward0>) |

As you can see, our custom model for Logistic Regression works exactly like the `nn.Sequential`

version above.

Putting everything together, the following is the complete code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
import torch torch.manual_seed(42) # build custom module for logistic regression class LogisticRegression(torch.nn.Module): # build the constructor def __init__(self, n_inputs): super(LogisticRegression, self).__init__() self.linear = torch.nn.Linear(n_inputs, 1) # make predictions def forward(self, x): y_pred = torch.sigmoid(self.linear(x)) return y_pred x = torch.tensor([[1], [2], [3], [4]], dtype=torch.float32) log_regr_cus = LogisticRegression(1) y_pred = log_regr_cus(x) print("here is model prediction: ", y_pred) |

## Summary

In this tutorial, you learned some basics of Logistic Regression and how it can be implemented in PyTorch. Particularly, you learned:

- How to make predictions with Logistic Regression in Pytroch.
- About the Logistic Function and its implementation on tensors.
- How to build a Logistic Regression model with
`nn.Sequential`

. - How to build a custom module for Logistic Regression.

How was this model trained before making predictions?

Hi Andrew…The code samples are an examples of how to make predictions once a model is trained. The following may be of interest:

https://machinelearningmastery.com/logistic-regression-tutorial-for-machine-learning/