If you’ve looked at Keras models on Github, you’ve probably noticed that there are some different ways to create models in Keras. There’s the Sequential model, which allows you to define an entire model in a single line, usually with some line breaks for readability. Then, there’s the functional interface that allows for more complicated model architectures, and there’s also the Model subclass which helps reusability. This article will explore the different ways to create models in Keras, along with their advantages and drawbacks. This will equip you with the knowledge you need to create your own machine learning models in Keras.
After you complete this tutorial, you will learn:
Different ways that Keras offers to build models
How to use the Sequential class, functional interface, and subclassing keras.Model to build Keras models
When to use the different methods to create Keras models
Let’s get started!
Three ways to build machine learning models in Keras Photo by Mike Szczepanski. Some rights reserved.
Overview
This tutorial is split into three parts, covering the different ways to build machine learning models in Keras:
Using the Sequential class
Using Keras’s functional interface
Subclassing keras.Model
Using the Sequential Class
The Sequential Model is just as the name implies. It consists of a sequence of layers, one after the other. From the Keras documentation,
“A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.”
It is a simple, easy-to-use way to start building your Keras model. To start, import Tensorflow and then the Sequential model:
1
2
import tensorflow astf
from tensorflow.keras import Sequential
Then, you can start building your machine learning model by stacking various layers together. For this example, let’s build a LeNet5 model with the classic CIFAR-10 image dataset as the input:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from tensorflow.keras.layers import Dense,Input,Flatten,Conv2D,MaxPool2D
Notice that you are just passing in an array of the layers you want your model to contain into the Sequential model constructor. Looking at the model.summary(), you can see the model’s architecture.
Now, let’s explore what the other ways of constructing Keras models can do, starting with the functional interface!
Using Keras’s Functional Interface
The next method of constructing Keras models you will explore uses Keras’s functional interface. The functional interface uses the layers as functions instead, taking in a Tensor and outputting a Tensor as well. The functional interface is a more flexible way of representing a Keras model as you are not restricted only to sequential models with layers stacked on top of one another. Instead, you can build models that branch into multiple paths, have multiple inputs, etc.
Consider an Add layer that takes inputs from two or more paths and adds the tensors together.
Add layer with two inputs
Since this cannot be represented as a linear stack of layers due to the multiple inputs, you are unable to define it using a Sequential object. Here’s where Keras’s functional interface comes in. You can define an Add layer with two input tensors as such:
1
2
from tensorflow.keras.layers import Add
add_layer=Add()([layer1,layer2])
Now that you’ve seen a quick example of the functional interface, let’s take a look at what the LeNet5 model that you defined by instantiating a Sequential class would look like using a functional interface.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import tensorflow astf
from tensorflow.keras.layers import Dense,Input,Flatten,Conv2D,MaxPool2D
As you can see, the model architecture is the same for both LeNet5 models you implemented using the functional interface or the Sequential class.
Now that you’ve seen how to use Keras’s functional interface, let’s look at a model architecture that you can implement using the functional interface but not with the Sequential class. For this example, look at the residual block introduced in ResNet. Visually, the residual block looks like this:
You can see that a model defined using the Sequential class would be unable to construct such a block due to the skip connection, which prevents this block from being represented as a simple stack of layers. Using the functional interface is one way you can define a ResNet block:
1
2
3
4
5
6
7
8
9
10
11
12
13
def residual_block(x,filters):
# store the input tensor to be added later as the identity
identity=x
# change the strides to do like pooling layer (need to see whether we connect before or after this layer though)
Keras also provides an object-oriented approach to creating models, which helps with reusability and allows you to represent the models you want to create as classes. This representation might be more intuitive since you can think about models as a set of layers strung together to form your network.
To begin subclassing keras.Model, you first need to import it:
1
from tensorflow.keras.models import Model
Then, you can start subclassing keras.Model. First, you need to build the layers that you want to use in your method calls since you only want to instantiate these layers once instead of each time you call your model. To keep in line with previous examples, let’s build a LeNet5 model here as well.
Then, override the call method to define what happens when the model is called. You override it with your model, which uses the layers you have built in the initializer.
1
2
3
4
5
6
7
8
9
10
11
12
13
def call(self,input_tensor):
# don't create layers here, need to create the layers in initializer,
# otherwise you will get the tf.Variable can only be created once error
conv1=self.conv1(input_tensor)
maxpool1=self.max_pool2x2(conv1)
conv2=self.conv2(maxpool1)
maxpool2=self.max_pool2x2(conv2)
conv3=self.conv3(maxpool2)
flatten=self.flatten(conv3)
fc2=self.fc2(flatten)
fc3=self.fc3(fc2)
returnfc3
It is important to have all the layers created at the class constructor, not inside the call() method. This is because the call() method will be invoked multiple times with different input tensors. But you want to use the same layer objects in each call to optimize their weight. You can then instantiate your new LeNet5 class and use it as part of a model:
1
2
3
4
5
6
input_layer=Input(shape=(32,32,3,))
x=LeNet5()(input_layer)
model=Model(inputs=input_layer,outputs=x)
print(model.summary(expand_nested=True))
And you can see that the model has the same number of parameters as the previous two versions of LeNet5 that were built previously and has the same structure within it.
In this post, you have seen three different ways to create models in Keras. In particular, this includes using the Sequential class, functional interface, and subclassing keras.Model. You have also seen examples of the same LeNet5 model being built using the different methods and a use case that can be done using the functional interface but not with the Sequential class.
Specifically, you learned:
Different ways that Keras offers to build models
How to use the Sequential class, functional interface, and subclassing keras.Model to build Keras models
When to use the different methods to create Keras models
Is there any way to make Sequential models with lots of layers? For example, it is very difficult to add 50 layers in a keras sequential model by hand? I was thinking about to make a keras model with 50 layers with 4 neours in each layer. Is there any recommendation to make it easier instead of writing 50 lines Dense layer in python?
Hi Jason,
Is there any way to make Sequential models with lots of layers? For example, it is very difficult to add 50 layers in a keras sequential model by hand? I was thinking about to make a keras model with 50 layers with 4 neours in each layer. Is there any recommendation to make it easier instead of writing 50 lines Dense layer in python?
Thanks in advance,
Faraz