Three Ways to Build Machine Learning Models in Keras

If you’ve looked at Keras models on Github, you’ve probably noticed that there are some different ways to create models in Keras. There’s the Sequential model, which allows you to define an entire model in a single line, usually with some line breaks for readability. Then, there’s the functional interface that allows for more complicated model architectures, and there’s also the Model subclass which helps reusability. This article will explore the different ways to create models in Keras, along with their advantages and drawbacks. This will equip you with the knowledge you need to create your own machine learning models in Keras.

After you complete this tutorial, you will learn:

• Different ways that Keras offers to build models
• How to use the Sequential class, functional interface, and subclassing keras.Model to build Keras models
• When to use the different methods to create Keras models

Let’s get started!

Three ways to build machine learning models in Keras
Photo by Mike Szczepanski. Some rights reserved.

Overview

This tutorial is split into three parts, covering the different ways to build machine learning models in Keras:

• Using the Sequential class
• Using Keras’s functional interface
• Subclassing keras.Model

Using the Sequential Class

The Sequential Model is just as the name implies. It consists of a sequence of layers, one after the other. From the Keras documentation,

“A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.”

It is a simple, easy-to-use way to start building your Keras model. To start, import Tensorflow and then the Sequential model:

Then, you can start building your machine learning model by stacking various layers together. For this example, let’s build a LeNet5 model with the classic CIFAR-10 image dataset as the input:

Notice that you are just passing in an array of the layers you want your model to contain into the Sequential model constructor. Looking at the model.summary(), you can see the model’s architecture.

And just to test out the model, let’s go ahead and load the CIFAR-10 dataset and run model.compile and model.fit:

This gives us this output:

That’s pretty good for a first pass at a model. Putting the code for LeNet5 using a Sequential model together, you have:

Now, let’s explore what the other ways of constructing Keras models can do, starting with the functional interface!

Using Keras’s Functional Interface

The next method of constructing Keras models you will explore uses Keras’s functional interface. The functional interface uses the layers as functions instead, taking in a Tensor and outputting a Tensor as well. The functional interface is a more flexible way of representing a Keras model as you are not restricted only to sequential models with layers stacked on top of one another. Instead, you can build models that branch into multiple paths, have multiple inputs, etc.

Consider an Add layer that takes inputs from two or more paths and adds the tensors together.

Add layer with two inputs

Since this cannot be represented as a linear stack of layers due to the multiple inputs, you are unable to define it using a Sequential object. Here’s where Keras’s functional interface comes in. You can define an Add layer with two input tensors as such:

Now that you’ve seen a quick example of the functional interface, let’s take a look at what the LeNet5 model that you defined by instantiating a Sequential class would look like using a functional interface.

And looking at the model summary:

As you can see, the model architecture is the same for both LeNet5 models you implemented using the functional interface or the Sequential class.

Now that you’ve seen how to use Keras’s functional interface, let’s look at a model architecture that you can implement using the functional interface but not with the Sequential class. For this example, look at the residual block introduced in ResNet. Visually, the residual block looks like this:

Residual block, source: https://arxiv.org/pdf/1512.03385.pdf

You can see that a model defined using the Sequential class would be unable to construct such a block due to the skip connection, which prevents this block from being represented as a simple stack of layers. Using the functional interface  is one way you can define a ResNet block:

Then, you can build a simple network using these residual blocks using the functional interface:

Running this code and looking at the model summary and training results:

And combining the code for our simple network using residual blocks:

Subclassing keras.Model

Keras also provides an object-oriented approach to creating models, which helps with reusability and allows you to represent the models you want to create as classes. This representation might be more intuitive since you can think about models as a set of layers strung together to form your network.

To begin subclassing keras.Model, you first need to import it:

Then, you can start subclassing keras.Model. First, you need to build the layers that you want to use in your method calls since you only want to instantiate these layers once instead of each time you call your model. To keep in line with previous examples, let’s build a LeNet5 model here as well.

Then, override the call method to define what happens when the model is called. You override it with your model, which uses the layers you have built in the initializer.

It is important to have all the layers created at the class constructor, not inside the call() method. This is because the call() method will be invoked multiple times with different input tensors. But you want to use the same layer objects in each call to optimize their weight. You can then instantiate your new LeNet5 class and use it as part of a model:

And you can see that the model has the same number of parameters as the previous two versions of LeNet5 that were built previously and has the same structure within it.

Combining all the code to create your LeNet5 subclass of keras.Model:

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Papers:

• Deep Residual Learning for Image Recognition (the ResNet paper)

APIs:

• Keras Sequential class
• Keras functional API
• Making new layers and models via subclassing

Summary

In this post, you have seen three different ways to create models in Keras. In particular, this includes using the Sequential class, functional interface, and subclassing keras.Model. You have also seen examples of the same LeNet5 model being built using the different methods and a use case that can be done using the functional interface but not with the Sequential class.

Specifically, you learned:

• Different ways that Keras offers to build models
• How to use the Sequential class, functional interface, and subclassing keras.Model to build Keras models
• When to use the different methods to create Keras models