How to Visualize Filters and Feature Maps in Convolutional Neural Networks

Deep learning neural networks are generally opaque, meaning that although they can make useful and skillful predictions, it is not clear how or why a given prediction was made.

Convolutional neural networks, have internal structures that are designed to operate upon two-dimensional image data, and as such preserve the spatial relationships for what was learned by the model. Specifically, the two-dimensional filters learned by the model can be inspected and visualized to discover the types of features that the model will detect, and the activation maps output by convolutional layers can be inspected to understand exactly what features were detected for a given input image.

In this tutorial, you will discover how to develop simple visualizations for filters and feature maps in a convolutional neural network.

After completing this tutorial, you will know:

  • How to develop a visualization for specific filters in a convolutional neural network.
  • How to develop a visualization for specific feature maps in a convolutional neural network.
  • How to systematically visualize feature maps for each block in a deep convolutional neural network.

Let’s get started.

How to Visualize Filters and Feature Maps in Convolutional Neural Networks

How to Visualize Filters and Feature Maps in Convolutional Neural Networks
Photo by Mark Kent, some rights reserved.

Tutorial Overview

This tutorial is divided into four parts; they are:

  1. Visualizing Convolutional Layers
  2. Pre-fit VGG Model
  3. How to Visualize Filters
  4. How to Visualize Feature Maps

Visualizing Convolutional Layers

Neural network models are generally referred to as being opaque. This means that they are poor at explaining the reason why a specific decision or prediction was made.

Convolutional neural networks are designed to work with image data, and their structure and function suggest that should be less inscrutable than other types of neural networks.

Specifically, the models are comprised of small linear filters and the result of applying filters called activation maps, or more generally, feature maps.

Both filters and feature maps can be visualized.

For example, we can design and understand small filters, such as line detectors. Perhaps visualizing the filters within a learned convolutional neural network can provide insight into how the model works.

The feature maps that result from applying filters to input images and to feature maps output by prior layers could provide insight into the internal representation that the model has of a specific input at a given point in the model.

We will explore both of these approaches to visualizing a convolutional neural network in this tutorial.

Want Results with Deep Learning for Computer Vision?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Download Your FREE Mini-Course

Pre-fit VGG Model

We need a model to visualize.

Instead of fitting a model from scratch, we can use a pre-fit prior state-of-the-art image classification model.

Keras provides many examples of well-performing image classification models developed by different research groups for the ImageNet Large Scale Visual Recognition Challenge, or ILSVRC. One example is the VGG-16 model that achieved top results in the 2014 competition.

This is a good model to use for visualization because it has a simple uniform structure of serially ordered convolutional and pooling layers, it is deep with 16 learned layers, and it performed very well, meaning that the filters and resulting feature maps will capture useful features. For more information on this model, see the 2015 paper “Very Deep Convolutional Networks for Large-Scale Image Recognition.”

We can load and summarize the VGG16 model with just a few lines of code; for example:

Running the example will load the model weights into memory and print a summary of the loaded model.

If this is the first time that you have loaded the model, the weights will be downloaded from the internet and stored in your home directory. These weights are approximately 500 megabytes and may take a moment to download depending on the speed of your internet connection.

We can see that the layers are well named, organized into blocks, and named with integer indexes within each block.

Now that we have a pre-fit model, we can use it as the basis for visualizations.

How to Visualize Filters

Perhaps the simplest visualization to perform is to plot the learned filters directly.

In neural network terminology, the learned filters are simply weights, yet because of the specialized two-dimensional structure of the filters, the weight values have a spatial relationship to each other and plotting each filter as a two-dimensional image is meaningful (or could be).

The first step is to review the filters in the model, to see what we have to work with.

The model summary printed in the previous section summarizes the output shape of each layer, e.g. the shape of the resulting feature maps. It does not give any idea of the shape of the filters (weights) in the network, only the total number of weights per layer.

We can access all of the layers of the model via the model.layers property.

Each layer has a layer.name property, where the convolutional layers have a naming convolution like block#_conv#, where the ‘#‘ is an integer. Therefore, we can check the name of each layer and skip any that don’t contain the string ‘conv‘.

Each convolutional layer has two sets of weights.

One is the block of filters and the other is the block of bias values. These are accessible via the layer.get_weights() function. We can retrieve these weights and then summarize their shape.

Tying this together, the complete example of summarizing the model filters is listed below.

Running the example prints a list of layer details including the layer name and the shape of the filters in the layer.

We can see that all convolutional layers use 3×3 filters, which are small and perhaps easy to interpret.

An architectural concern with a convolutional neural network is that the depth of a filter must match the depth of the input for the filter (e.g. the number of channels).

We can see that for the input image with three channels for red, green and blue, that each filter has a depth of three (here we are working with a channel-last format). We could visualize one filter as a plot with three images, one for each channel, or compress all three down to a single color image, or even just look at the first channel and assume the other channels will look the same. The problem is, we then have 63 other filters that we might like to visualize.

We can retrieve the filters from the first layer as follows:

The weight values will likely be small positive and negative values centered around 0.0.

We can normalize their values to the range 0-1 to make them easy to visualize.

Now we can enumerate the first six filters out of the 64 in the block and plot each of the three channels of each filter.

We use the matplotlib library and plot each filter as a new row of subplots, and each filter channel or depth as a new column.

Tying this together, the complete example of plotting the first six filters from the first hidden convolutional layer in the VGG16 model is listed below.

Running the example creates a figure with six rows of three images, or 18 images, one row for each filter and one column for each channel

We can see that in some cases, the filter is the same across the channels (the first row), and in others, the filters differ (the last row).

The dark squares indicate small or inhibitory weights and the light squares represent large or excitatory weights. Using this intuition, we can see that the filters on the first row detect a gradient from light in the top left to dark in the bottom right.

Plot of the First 6 Filters From VGG16 With One Subplot per Channel

Plot of the First 6 Filters From VGG16 With One Subplot per Channel

Although we have a visualization, we only see the first six of the 64 filters in the first convolutional layer. Visualizing all 64 filters in one image is feasible.

Sadly, this does not scale; if we wish to start looking at filters in the second convolutional layer, we can see that again we have 64 filters, but each has 64 channels to match the input feature maps. To see all 64 channels in a row for all 64 filters would require (64×64) 4,096 subplots in which it may be challenging to see any detail.

How to Visualize Feature Maps

The activation maps, called feature maps, capture the result of applying the filters to input, such as the input image or another feature map.

The idea of visualizing a feature map for a specific input image would be to understand what features of the input are detected or preserved in the feature maps. The expectation would be that the feature maps close to the input detect small or fine-grained detail, whereas feature maps close to the output of the model capture more general features.

In order to explore the visualization of feature maps, we need input for the VGG16 model that can be used to create activations. We will use a simple photograph of a bird. Specifically, a Robin, taken by Chris Heald and released under a permissive license.

Download the photograph and place it in your current working directory with the filename ‘bird.jpg‘.

Robin, by Chris Heald

Robin, by Chris Heald

Next, we need a clearer idea of the shape of the feature maps output by each of the convolutional layers and the layer index number so that we can retrieve the appropriate layer output.

The example below will enumerate all layers in the model and print the output size or feature map size for each convolutional layer as well as the layer index in the model.

Running the example, we see the same output shapes as we saw in the model summary, but in this case only for the convolutional layers.

We can use this information and design a new model that is a subset of the layers in the full VGG16 model. The model would have the same input layer as the original model, but the output would be the output of a given convolutional layer, which we know would be the activation of the layer or the feature map.

For example, after loading the VGG model, we can define a new model that outputs a feature map from the first convolutional layer (index 1) as follows.

Making a prediction with this model will give the feature map for the first convolutional layer for a given provided input image. Let’s implement this.

After defining the model, we need to load the bird image with the size expected by the model, in this case, 224×224.

Next, the image PIL object needs to be converted to a NumPy array of pixel data and expanded from a 3D array to a 4D array with the dimensions of [samples, rows, cols, channels], where we only have one sample.

The pixel values then need to be scaled appropriately for the VGG model.

We are now ready to get the feature map. We can do this easy by calling the model.predict() function and passing in the prepared single image.

We know the result will be a feature map with 224x224x64. We can plot all 64 two-dimensional images as an 8×8 square of images.

Tying all of this together, the complete code example of visualizing the feature map for the first convolutional layer in the VGG16 model for a bird input image is listed below.

Running the example first summarizes the new, smaller model that takes an image and outputs a feature map.

Remember: this model is much smaller than the VGG16 model, but still uses the same weights (filters) in the first convolutional layer as the VGG16 model.

Next, a figure is created that shows all 64 feature maps as subplots.

We can see that the result of applying the filters in the first convolutional layer is a lot of versions of the bird image with different features highlighted.

For example, some highlight lines, other focus on the background or the foreground.

Visualization of the Feature Maps Extracted From the First Convolutional Layer in the VGG16 Model

Visualization of the Feature Maps Extracted From the First Convolutional Layer in the VGG16 Model

This is an interesting result and generally matches our expectation. We could update the example to plot the feature maps from the output of other specific convolutional layers.

Another approach would be to collect feature maps output from each block of the model in a single pass, then create an image of each.

There are five main blocks in the image (e.g. block1, block2, etc.) that end in a pooling layer. The layer indexes of the last convolutional layer in each block are [2, 5, 9, 13, 17].

We can define a new model that has multiple outputs, one feature map output for each of the last convolutional layer in each block; for example:

Making a prediction with this new model will result in a list of feature maps.

We know that the number of feature maps (e.g. depth or number of channels) in deeper layers is much more than 64, such as 256 or 512. Nevertheless, we can cap the number of feature maps visualized at 64 for consistency.

Tying these changes together, we can now create five separate plots for each of the five blocks in the VGG16 model for our bird photograph. The complete listing is provided below.

Running the example results in five plots showing the feature maps from the five main blocks of the VGG16 model.

We can see that the feature maps closer to the input of the model capture a lot of fine detail in the image and that as we progress deeper into the model, the feature maps show less and less detail.

This pattern was to be expected, as the model abstracts the features from the image into more general concepts that can be used to make a classification. Although it is not clear from the final image that the model saw a bird, we generally lose the ability to interpret these deeper feature maps.

Visualization of the Feature Maps Extracted From Block 1 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 1 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 2 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 2 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 3 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 3 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 4 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 4 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 5 in the VGG16 Model

Visualization of the Feature Maps Extracted From Block 5 in the VGG16 Model

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Books

API

Articles

Summary

In this tutorial, you discovered how to develop simple visualizations for filters and feature maps in a convolutional neural network.

Specifically, you learned:

  • How to develop a visualization for specific filters in a convolutional neural network.
  • How to develop a visualization for specific feature maps in a convolutional neural network.
  • How to systematically visualize feature maps for each block in a deep convolutional neural network.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.


Develop Deep Learning Models for Vision Today!

Deep Learning for Computer Vision

Develop Your Own Vision Models in Minutes

…with just a few lines of python code

Discover how in my new Ebook:
Deep Learning for Computer Vision

It provides self-study tutorials on topics like: classification, object detection (yolo and rcnn), face recognition (vggface and facenet), data preparation and much more…

Finally Bring Deep Learning to your Vision Projects

Skip the Academics. Just Results.

Click to learn more.


21 Responses to How to Visualize Filters and Feature Maps in Convolutional Neural Networks

  1. Pepe May 6, 2019 at 7:26 pm #

    Thanks a lot, sir!! It helps my thesis manuscript in finding feature maps in each layer in my model. By the way, I have a question sir about how to display in values (not an image) in each filters and biases in every convolutional layer in my CNN model?

    • Jason Brownlee May 7, 2019 at 6:15 am #

      I’m happy to hear that.

      You could print the arrays and inspect the values.

  2. Pepe May 7, 2019 at 12:07 pm #

    Okay sir thank you 🙂

  3. Pepe May 7, 2019 at 12:16 pm #

    Sir can I make a request, I would like to display every feature maps in every convolutional layer in my model and I have a problem of my code in accessing all feature maps in every layer. Any suggestion sir? I hope your response sir and thank you.

  4. Pepe May 7, 2019 at 12:43 pm #

    index = [0, 1, 6, 9, 14, 17]
    outputs = [self.model.layers[i].output for i in index]
    self.model = Model(inputs=self.model.inputs, outputs=outputs)
    featureMaps = self.model.predict(self.testImage)

    print((np.shape(featureMaps[0][0])))
    print(“FeatureMapsLen: “+str(np.shape(featureMaps[0]))[2])
    numOfFeaturemaps = (np.shape(featureMaps[0][0]))[2]

    print(“numOfFeatureMaps: “+str(numOfFeaturemaps)

    fig=plt.figure(figsize=(16,16))
    subplotNum=int(np.ceil(np.sqrt(numOfFeaturemaps)))
    for i in range(int(numOfFeaturemaps)):
    idx = fig.add_subplot(subplotNum, subplotNum, i+1)
    idx.imshow(featureMaps[0, :, :, i], cmap=’viridis’) #I’m have been stack here for long sir jason!!
    plt.xticks(np.array([]))
    plt.yticks(np.array([]))
    plt.tight_layout()

    plt.savefig(“featureMaps/featuremaps@Layer{}”.format(self.layerNum) + ‘.png’)
    outputImg = QtGui.QPixmap(“featureMaps/featuremaps@Layer{}”.format(self.layerNum) + ‘.png’)
    self.userInterface.labelImageContainer.setScaledContents(False)#Fixed display
    self.userInterface.labelImageContainer.setPixmap(outputImg)

  5. christian May 7, 2019 at 4:05 pm #

    LiME is known to “explain” the results of a classification problem. Can we use the filters that you have explained in order to explain the results of a segmentation problem?

    • Jason Brownlee May 8, 2019 at 6:41 am #

      Perhaps. I have not seen any work on the topic.

      Perhaps try searching on scholar.google.com.

  6. Shah May 7, 2019 at 4:29 pm #

    great explanation. The best part is step by step explanation with code. Really helpful 🙂

  7. Hamed May 10, 2019 at 8:58 am #

    For no apparent reason, I have my Keras via TensorFlow so I have to modify, for instance, this line of code:
    “from keras.applications.vgg16 import VGG16”
    to
    “from tensordlow.keras.applications.vgg16 import VGG16”
    and when I loaded it for the first time it showed it was downloading from Github but then it is now training! Is this normal?

    Thanks!

    • Jason Brownlee May 10, 2019 at 1:41 pm #

      It is downloaded the first time. That is normal.

      • Hamed May 11, 2019 at 6:42 am #

        Right! But I had a difficulty to download it and after 33Mb I got disconnection from remote server error so I opened vgg16.py located in my tensorflow/python/keras/applications, got the link, downloaded manually and change the default from ‘imagenet’ to the path that the manually downloaded file was located (I transferred it to ‘applications’ folder but still asked me the whole path and not only the filename: ‘vgg16_weights_tf_dim_ordering_tf_kernels.h5’) so it worked out for me. Would you mind if you kindly tell me why it asked the whole path? It should recognized its current path as it’s running in it.

        • Jason Brownlee May 12, 2019 at 6:36 am #

          I don’t know why it asked for the full path, sorry.

  8. Mike May 10, 2019 at 12:24 pm #

    Thanks for the lesson!!

    I would love to see a detailed description of how to create class activation maps.

    I’ve been using some of the code from your books to train a CNN to recognize tears of the anterior crucial efforts ligament. I’d love to see just what parts of the image my model is using to make its decisions.

    • Mike May 10, 2019 at 12:25 pm #

      Anterior cruciate ligament, that is.

    • Jason Brownlee May 10, 2019 at 1:42 pm #

      Great suggestion, thanks.

      Well done on your application Mike!

      • M.LABENI May 11, 2019 at 7:46 am #

        Thanks, that’s very kind of you .

  9. ganga May 17, 2019 at 3:29 am #

    Hello Jason,

    Thanks for your code..It really helpful.
    I have a query here.
    1. We are using single image in this model instead is it possible to use batch of images to visualise them in the model ?
    2. I am a newbie. Don’t mind me for a silly question here please. Is it possible to view outputs at fully connected layers like Conv layers we did here?
    3. Is there any limit on number of neurons used in dense layers?

    • Jason Brownlee May 17, 2019 at 5:59 am #

      You would process one image at a time, if you are only looking at activation maps.

      You can review activations of any layer, but Dense layers will not form images, it will be noise.

      The only limit is the mount of RAM you have.

Leave a Reply