Transfer Learning in Keras with Computer Vision Models

Last Updated on

Deep convolutional neural network models may take days or even weeks to train on very large datasets.

A way to short-cut this process is to re-use the model weights from pre-trained models that were developed for standard computer vision benchmark datasets, such as the ImageNet image recognition tasks. Top performing models can be downloaded and used directly, or integrated into a new model for your own computer vision problems.

In this post, you will discover how to use transfer learning when developing convolutional neural networks for computer vision applications.

After reading this post, you will know:

  • Transfer learning involves using models trained on one problem as a starting point on a related problem.
  • Transfer learning is flexible, allowing the use of pre-trained models directly, as feature extraction preprocessing, and integrated into entirely new models.
  • Keras provides convenient access to many top performing models on the ImageNet image recognition tasks such as VGG, Inception, and ResNet.

Discover how to build models for photo classification, object detection, face recognition, and more in my new computer vision book, with 30 step-by-step tutorials and full source code.

Let’s get started.

How to Use Transfer Learning when Developing Convolutional Neural Network Models

How to Use Transfer Learning when Developing Convolutional Neural Network Models
Photo by GoToVan, some rights reserved.


This tutorial is divided into five parts; they are:

  1. What Is Transfer Learning?
  2. Transfer Learning for Image Recognition
  3. How to Use Pre-Trained Models
  4. Models for Transfer Learning
  5. Examples of Using Pre-Trained Models

What Is Transfer Learning?

Transfer learning generally refers to a process where a model trained on one problem is used in some way on a second related problem.

In deep learning, transfer learning is a technique whereby a neural network model is first trained on a problem similar to the problem that is being solved. One or more layers from the trained model are then used in a new model trained on the problem of interest.

This is typically understood in a supervised learning context, where the input is the same but the target may be of a different nature. For example, we may learn about one set of visual categories, such as cats and dogs, in the first setting, then learn about a different set of visual categories, such as ants and wasps, in the second setting.

— Page 536, Deep Learning, 2016.

Transfer learning has the benefit of decreasing the training time for a neural network model and can result in lower generalization error.

The weights in re-used layers may be used as the starting point for the training process and adapted in response to the new problem. This usage treats transfer learning as a type of weight initialization scheme. This may be useful when the first related problem has a lot more labeled data than the problem of interest and the similarity in the structure of the problem may be useful in both contexts.

… the objective is to take advantage of data from the first setting to extract information that may be useful when learning or even when directly making predictions in the second setting.

— Page 538, Deep Learning, 2016.

Want Results with Deep Learning for Computer Vision?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Download Your FREE Mini-Course

Transfer Learning for Image Recognition

A range of high-performing models have been developed for image classification and demonstrated on the annual ImageNet Large Scale Visual Recognition Challenge, or ILSVRC.

This challenge, often referred to simply as ImageNet, given the source of the image used in the competition, has resulted in a number of innovations in the architecture and training of convolutional neural networks. In addition, many of the models used in the competitions have been released under a permissive license.

These models can be used as the basis for transfer learning in computer vision applications.

This is desirable for a number of reasons, not least:

  • Useful Learned Features: The models have learned how to detect generic features from photographs, given that they were trained on more than 1,000,000 images for 1,000 categories.
  • State-of-the-Art Performance: The models achieved state of the art performance and remain effective on the specific image recognition task for which they were developed.
  • Easily Accessible: The model weights are provided as free downloadable files and many libraries provide convenient APIs to download and use the models directly.

The model weights can be downloaded and used in the same model architecture using a range of different deep learning libraries, including Keras.

How to Use Pre-Trained Models

The use of a pre-trained model is limited only by your creativity.

For example, a model may be downloaded and used as-is, such as embedded into an application and used to classify new photographs.

Alternately, models may be downloaded and use as feature extraction models. Here, the output of the model from a layer prior to the output layer of the model is used as input to a new classifier model.

Recall that convolutional layers closer to the input layer of the model learn low-level features such as lines, that layers in the middle of the layer learn complex abstract features that combine the lower level features extracted from the input, and layers closer to the output interpret the extracted features in the context of a classification task.

Armed with this understanding, a level of detail for feature extraction from an existing pre-trained model can be chosen. For example, if a new task is quite different from classifying objects in photographs (e.g. different to ImageNet), then perhaps the output of the pre-trained model after the few layers would be appropriate. If a new task is quite similar to the task of classifying objects in photographs, then perhaps the output from layers much deeper in the model can be used, or even the output of the fully connected layer prior to the output layer can be used.

The pre-trained model can be used as a separate feature extraction program, in which case input can be pre-processed by the model or portion of the model to a given an output (e.g. vector of numbers) for each input image, that can then use as input when training a new model.

Alternately, the pre-trained model or desired portion of the model can be integrated directly into a new neural network model. In this usage, the weights of the pre-trained can be frozen so that they are not updated as the new model is trained. Alternately, the weights may be updated during the training of the new model, perhaps with a lower learning rate, allowing the pre-trained model to act like a weight initialization scheme when training the new model.

We can summarize some of these usage patterns as follows:

  • Classifier: The pre-trained model is used directly to classify new images.
  • Standalone Feature Extractor: The pre-trained model, or some portion of the model, is used to pre-process images and extract relevant features.
  • Integrated Feature Extractor: The pre-trained model, or some portion of the model, is integrated into a new model, but layers of the pre-trained model are frozen during training.
  • Weight Initialization: The pre-trained model, or some portion of the model, is integrated into a new model, and the layers of the pre-trained model are trained in concert with the new model.

Each approach can be effective and save significant time in developing and training a deep convolutional neural network model.

It may not be clear as to which usage of the pre-trained model may yield the best results on your new computer vision task, therefore some experimentation may be required.

Models for Transfer Learning

There are perhaps a dozen or more top-performing models for image recognition that can be downloaded and used as the basis for image recognition and related computer vision tasks.

Perhaps three of the more popular models are as follows:

  • VGG (e.g. VGG16 or VGG19).
  • GoogLeNet (e.g. InceptionV3).
  • Residual Network (e.g. ResNet50).

These models are both widely used for transfer learning both because of their performance, but also because they were examples that introduced specific architectural innovations, namely consistent and repeating structures (VGG), inception modules (GoogLeNet), and residual modules (ResNet).

Keras provides access to a number of top-performing pre-trained models that were developed for image recognition tasks.

They are available via the Applications API, and include functions to load a model with or without the pre-trained weights, and prepare data in a way that a given model may expect (e.g. scaling of size and pixel values).

The first time a pre-trained model is loaded, Keras will download the required model weights, which may take some time given the speed of your internet connection. Weights are stored in the .keras/models/ directory under your home directory and will be loaded from this location the next time that they are used.

When loading a given model, the “include_top” argument can be set to False, in which case the fully-connected output layers of the model used to make predictions is not loaded, allowing a new output layer to be added and trained. For example:

Additionally, when the “include_top” argument is False, the “input_tensor” argument must be specified, allowing the expected fixed-sized input of the model to be changed. For example:

A model without a top will output activations from the last convolutional or pooling layer directly. One approach to summarizing these activations for thier use in a classifier or as a feature vector representation of input is to add a global pooling layer, such as a max global pooling or average global pooling. The result is a vector that can be used as a feature descriptor for an input. Keras provides this capability directly via the ‘pooling‘ argument that can be set to ‘avg‘ or ‘max‘. For example:

Images can be prepared for a given model using the preprocess_input() function; e.g., pixel scaling is performed in a way that was performed to images in the training dataset when the model was developed. For example:

Finally, you may wish to use a model architecture on your dataset, but not use the pre-trained weights, and instead initialize the model with random weights and train the model from scratch.

This can be achieved by setting the ‘weights‘ argument to None instead of the default ‘imagenet‘. Additionally, the ‘classes‘ argument can be set to define the number of classes in your dataset, which will then be configured for you in the output layer of the model. For example:

Now that we are familiar with the API, let’s take a look at loading three models using the Keras Applications API.

Load the VGG16 Pre-trained Model

The VGG16 model was developed by the Visual Graphics Group (VGG) at Oxford and was described in the 2014 paper titled “Very Deep Convolutional Networks for Large-Scale Image Recognition.”

By default, the model expects color input images to be rescaled to the size of 224×224 squares.

The model can be loaded as follows:

Running the example will load the VGG16 model and download the model weights if required.

The model can then be used directly to classify a photograph into one of 1,000 classes. In this case, the model architecture is summarized to confirm that it was loaded correctly.

Load the InceptionV3 Pre-Trained Model

The InceptionV3 is the third iteration of the inception architecture, first developed for the GoogLeNet model.

This model was developed by researchers at Google and described in the 2015 paper titled “Rethinking the Inception Architecture for Computer Vision.”

The model expects color images to have the square shape 299×299.

The model can be loaded as follows:

Running the example will load the model, downloading the weights if required, and then summarize the model architecture to confirm it was loaded correctly.

The output is omitted in this case for brevity, as it is a deep model with many layers.

Load the ResNet50 Pre-trained Model

The Residual Network, or ResNet for short, is a model that makes use of the residual module involving shortcut connections.

It was developed by researchers at Microsoft and described in the 2015 paper titled “Deep Residual Learning for Image Recognition.”

The model expects color images to have the square shape 224×224.

Running the example will load the model, downloading the weights if required, and then summarize the model architecture to confirm it was loaded correctly.

The output is omitted in this case for brevity, as it is a deep model.

Examples of Using Pre-Trained Models

Now that we are familiar with how to load pre-trained models in Keras, let’s look at some examples of how they might be used in practice.

In these examples, we will work with the VGG16 model as it is a relatively straightforward model to use and a simple model architecture to understand.

We also need a photograph to work with in these examples. Below is a photograph of a dog, taken by Justin Morgan and made available under a permissive license.

Photograph of a Dog

Photograph of a Dog

Download the photograph and place it in your current working directory with the filename ‘dog.jpg‘.

Pre-Trained Model as Classifier

A pre-trained model can be used directly to classify new photographs as one of the 1,000 known classes in the image classification task in the ILSVRC.

We will use the VGG16 model to classify new images.

First, the photograph needs to loaded and reshaped to a 224×224 square, expected by the model, and the pixel values scaled in the way expected by the model. The model operates on an array of samples, therefore the dimensions of a loaded image need to be expanded by 1, for one image with 224×224 pixels and three channels.

Next, the model can be loaded and a prediction made.

This means that a predicted probability of the photo belonging to each of the 1,000 classes is made. In this example, we are only concerned with the most likely class, so we can decode the predictions and retrieve the label or name of the class with the highest probability.

Tying all of this together, the complete example below loads a new photograph and predicts the most likely class.

Running the example predicts more than just dog; it also predicts the specific breed of ‘Doberman‘ with a probability of 33.59%, which may, in fact, be correct.

Pre-Trained Model as Feature Extractor Preprocessor

The pre-trained model may be used as a standalone program to extract features from new photographs.

Specifically, the extracted features of a photograph may be a vector of numbers that the model will use to describe the specific features in a photograph. These features can then be used as input in the development of a new model.

The last few layers of the VGG16 model are fully connected layers prior to the output layer. These layers will provide a complex set of features to describe a given input image and may provide useful input when training a new model for image classification or related computer vision task.

The image can be loaded and prepared for the model, as we did before in the previous example.

We will load the model with the classifier output part of the model, but manually remove the final output layer. This means that the second last fully connected layer with 4,096 nodes will be the new output layer.

This vector of 4,096 numbers will be used to represent the complex features of a given input image that can then be saved to file to be loaded later and used as input to train a new model. We can save it as a pickle file.

Tying all of this together, the complete example of using the model as a standalone feature extraction model is listed below.

Running the example loads the photograph, then prepares the model as a feature extraction model.

The features are extracted from the loaded photo and the shape of the feature vector is printed, showing it has 4,096 numbers. This feature is then saved to a new file dog.pkl in the current working directory.

This process could be repeated for each photo in a new training dataset.

Pre-Trained Model as Feature Extractor in Model

We can use some or all of the layers in a pre-trained model as a feature extraction component of a new model directly.

This can be achieved by loading the model, then simply adding new layers. This may involve adding new convolutional and pooling layers to expand upon the feature extraction capabilities of the model or adding new fully connected classifier type layers to learn how to interpret the extracted features on a new dataset, or some combination.

For example, we can load the VGG16 models without the classifier part of the model by specifying the “include_top” argument to “False”, and specify the preferred shape of the images in our new dataset as 300×300.

We can then use the Keras function API to add a new Flatten layer after the last pooling layer in the VGG16 model, then define a new classifier model with a Dense fully connected layer and an output layer that will predict the probability for 10 classes.

An alternative approach to adding a Flatten layer would be to define the VGG16 model with an average pooling layer, and then add fully connected layers. Perhaps try both approaches on your application and see which results in the best performance.

The weights of the VGG16 model and the weights for the new model will all be trained together on the new dataset.

The complete example is listed below.

Running the example defines the new model ready for training and summarizes the model architecture.

We can see that we have flattened the output of the last pooling layer and added our new fully connected layers.

Alternately, we may wish to use the VGG16 model layers, but train the new layers of the model without updating the weights of the VGG16 layers. This will allow the new output layers to learn to interpret the learned features of the VGG16 model.

This can be achieved by setting the “trainable” property on each of the layers in the loaded VGG model to False prior to training. For example:

You can pick and choose which layers are trainable.

For example, perhaps you want to retrain some of the convolutional layers deep in the model, but none of the layers earlier in the model. For example:

Further Reading

This section provides more resources on the topic if you are looking to go deeper.







In this post, you discovered how to use transfer learning when developing convolutional neural networks for computer vision applications.

Specifically, you learned:

  • Transfer learning involves using models trained on one problem as a starting point on a related problem.
  • Transfer learning is flexible, allowing the use of pre-trained models directly as feature extraction preprocessing and integrated into entirely new models.
  • Keras provides convenient access to many top performing models on the ImageNet image recognition tasks such as VGG, Inception, and ResNet.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Develop Deep Learning Models for Vision Today!

Deep Learning for Computer Vision

Develop Your Own Vision Models in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Deep Learning for Computer Vision

It provides self-study tutorials on topics like:
classification, object detection (yolo and rcnn), face recognition (vggface and facenet), data preparation and much more...

Finally Bring Deep Learning to your Vision Projects

Skip the Academics. Just Results.

See What's Inside

94 Responses to Transfer Learning in Keras with Computer Vision Models

  1. Ludwig May 15, 2019 at 8:37 am #

    amazing post! thank you.

    • Jason Brownlee May 15, 2019 at 2:41 pm #

      Thanks, I’m glad it helped!

      • Andre October 1, 2019 at 9:15 pm #

        can I train one of the models with rectangular input image?, eg :input_shape=(100, 150, 3)

      • raphael June 18, 2020 at 7:28 am #

        great post

  2. Sujit Nalawade May 17, 2019 at 11:13 pm #

    best post i’ve read on transfer learning

  3. halima May 21, 2019 at 8:21 am #

    hello . is transfer learning just for image recognition?
    can i use it in another apps? . for example i have models using classification methods of machine learning but not deep learning or neural networks ,
    can i use transfer learning to have the knowledge from one model and use it as feature in another model ?
    and if i can how i will do this ?
    thanks a lot.

    • Jason Brownlee May 21, 2019 at 2:41 pm #

      No, transfer learning is also very common for NLP e.g. text data.

      You could use it for tabular data too, if you had enough data and pre-trained models.

  4. Atefeh July 22, 2019 at 6:27 pm #

    Hello Mr.Brownlee
    After saving the feature vector gained from the above example,is it possible to use these vectors directly as the inputs for a LSTM?

  5. Rakesh July 27, 2019 at 9:40 pm #

    well written, super helpful!

    Just a small typo – you may want to change to (300,300,3) in this line (first snippet under “Pre-Trained Model as Feature Extractor in Model”):
    model = VGG16(include_top=False, input_shape=(224, 224, 3))

  6. Gautam KIshore Shahi August 29, 2019 at 11:59 pm #


    I am writing to use the concept and the code mentioned for my data, data is of binary class after loading the image, when I click on the predict, I get the error, Uncaught (in promise) Error: Error when checking: expected tfjs@ flatten_1_input to have shape [null,7,7,512] but got array with shape [1,224,224,3].

    Can you please help me to solve this issue?

    • Jason Brownlee August 30, 2019 at 6:25 am #

      Sorry, I have not seen this issue.

      Perhaps try posting your code and error to stackoverflow?

  7. Jacques Thibodeau September 27, 2019 at 6:36 am #

    Does it make sense to use convolutional layers in the model you build on top of the pre-trained model? I’m looking for examples of this, but only really find fully connected layers after our pre-trained model. If we can, how should we go about it? If we can’t, why not?

    • Jason Brownlee September 27, 2019 at 8:06 am #

      It may, if you want to interpret the learned features by another model without messing them up via adjusting their weights.

      I may have an example, perhaps check some of the tutorials for image classification here:

      • Jacques Thibodeau September 27, 2019 at 11:02 pm #

        Yes, that makes sense!

        In the context of building a more accurate transfer learning model, it would make less sense? At the moment I’m working on the pneumonia Kaggle dataset while using InceptionV3 as the pre-trained model. The dataset and number of classes are quite small compared to imagenet. Would retraining some of the layers within Inception be a good idea?

        (I’m getting the impression that in this context I should just train the model, and then tune it based on what I get instead of getting too theoretical.)

        • Jason Brownlee September 28, 2019 at 6:18 am #

          I think starting with a pre-trained model is almost always the way to go, and tuning the output layers or adding some new layers and tuning them should be tried.

  8. Theekshana October 15, 2019 at 5:57 pm #

    Hi Jason,

    I’m planning to use Transfer learning for ECG signal related classification project. I’m thinking of using existing models in Keras API (since it’s difficult to find a signal related pre-trained model.)

    Talking about the data set, I have only 1000 signal samples. Therefore, now the transfer learning problem narrows down to “target dataset is small and different from the base training dataset” problem.

    Can you please provide me with some suggestions to approach this (books, research papers).

    Big fan of your website. Thanks.


    • Jason Brownlee October 16, 2019 at 7:58 am #

      Perhaps you can use a model from a related time series classification problem for transfer learning?

  9. sneh October 25, 2019 at 6:15 pm #

    can we use vgg16 which is trained on image dataset for an pima-Indian diabetes dataset?
    or can a image based Pretrained model can be used for non-image dataset?

  10. Dave October 27, 2019 at 9:31 pm #

    So with this approach, how would one go about training multiple classes?

    • Dave October 27, 2019 at 9:33 pm #

      Sorry, wrong article, please ignore…

  11. Atefeh November 8, 2019 at 5:55 pm #

    Hello Mr.Brownlee
    In the last code(Pre-Trained Model as Feature Extractor in Model),where does the feature vectors has been saved?
    I mean how can i use the feature vectors?
    for example if i have 3000 images which are from 10 classes(each class has 300 images),how can i have the feature vectors of each image in a class?

    in your last code, if we have 3000 images as the network input,then do we have 3000 feature vectors? and so if it is right, how we can access to those vectors?

    thank you

    • Jason Brownlee November 9, 2019 at 6:11 am #

      You can pass an image through the model to get the feature vectors for the image.

      You can repeat this for all your images and save the array of vectors to file if you wish.

  12. MonaST November 16, 2019 at 11:10 pm #

    I tried to fit the following model but there are an error and I did not know how to fix it. Could you help me , please.

    InvalidArgumentError Traceback (most recent call last)
    1 # train
    —-> 2, y_train, batch_size=32, epochs=1,validation_data=(X_test,y_test),shuffle=False,verbose=2)

    C:\Anaconda64\lib\site-packages\keras\engine\ in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    1237 steps_per_epoch=steps_per_epoch,
    1238 validation_steps=validation_steps,
    -> 1239 validation_freq=validation_freq)
    1241 def evaluate(self,

    C:\Anaconda64\lib\site-packages\keras\engine\ in fit_loop(model, fit_function, fit_inputs, out_labels, batch_size, epochs, verbose, callbacks, val_function, val_inputs, shuffle, initial_epoch, steps_per_epoch, validation_steps, validation_freq)
    194 ins_batch[i] = ins_batch[i].toarray()
    –> 196 outs = fit_function(ins_batch)
    197 outs = to_list(outs)
    198 for l, o in zip(out_labels, outs):

    C:\Anaconda64\lib\site-packages\tensorflow_core\python\keras\ in __call__(self, inputs)
    3738 value = math_ops.cast(value, tensor.dtype)
    3739 converted_inputs.append(value)
    -> 3740 outputs = self._graph_fn(*converted_inputs)
    3742 # EagerTensor.numpy() will often make a copy to ensure memory safety.

    C:\Anaconda64\lib\site-packages\tensorflow_core\python\eager\ in __call__(self, *args, **kwargs)
    1079 TypeError: For invalid positional/keyword argument combinations.
    1080 “””
    -> 1081 return self._call_impl(args, kwargs)
    1083 def _call_impl(self, args, kwargs, cancellation_manager=None):

    C:\Anaconda64\lib\site-packages\tensorflow_core\python\eager\ in _call_impl(self, args, kwargs, cancellation_manager)
    1119 raise TypeError(“Keyword arguments {} unknown. Expected {}.”.format(
    1120 list(kwargs.keys()), list(self._arg_keywords)))
    -> 1121 return self._call_flat(args, self.captured_inputs, cancellation_manager)
    1123 def _filtered_call(self, args, kwargs):

    C:\Anaconda64\lib\site-packages\tensorflow_core\python\eager\ in _call_flat(self, args, captured_inputs, cancellation_manager)
    1222 if executing_eagerly:
    1223 flat_outputs =
    -> 1224 ctx, args, cancellation_manager=cancellation_manager)
    1225 else:
    1226 gradient_name = self._delayed_rewrite_functions.register()

    C:\Anaconda64\lib\site-packages\tensorflow_core\python\eager\ in call(self, ctx, args, cancellation_manager)
    509 inputs=args,
    510 attrs=(“executor_type”, executor_type, “config_proto”, config),
    –> 511 ctx=ctx)
    512 else:
    513 outputs = execute.execute_with_cancellation(

    C:\Anaconda64\lib\site-packages\tensorflow_core\python\eager\ in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
    65 else:
    66 message = e.message
    —> 67 six.raise_from(core._status_to_exception(e.code, message), None)
    68 except TypeError as e:
    69 keras_symbolic_tensors = [

    C:\Anaconda64\lib\site-packages\ in raise_from(value, from_value)

    InvalidArgumentError: Matrix size-incompatible: In[0]: [1,802816], In[1]: [25088,1024]
    [[node dense_1/Relu (defined at C:\Anaconda64\lib\site-packages\tensorflow_core\python\framework\ ]] [Op:__inference_keras_scratch_graph_1052]

    Function call stack:

    My model is:

    load model without classifier layers without change imput size
    modelVGG = VGG16(include_top=False, input_shape=(224, 224, 3))
    # add new classifier layers
    flat1 = Flatten()(modelVGG.outputs)
    class1 = Dense(1024, activation=’relu’)(flat1)
    output = Dense(49, activation=’sigmoid’)(class1)
    # define new model
    model = Model(inputs=modelVGG.inputs, outputs=output)
    # mark loaded layers as not trainable
    for layer in modelVGG.layers:
    layer.trainable = Falsee


    (1452, 224, 224, 3)
    (1452, 49)
    (435, 224, 224, 3)
    (435, 49)

  13. Hadi November 19, 2019 at 8:13 pm #

    amazing post!
    Thank you for your time.

  14. shazia saqib November 22, 2019 at 1:23 am #

    you are a great support

  15. saif November 24, 2019 at 6:57 am #

    can i use pretrained model to facial emotion recognition with fer2013 dataset?

  16. fds December 7, 2019 at 5:58 pm #

    is it possible to concatenate multiple pretrained model and used for feature extraction in image classification…? does it make sense?

    • Jason Brownlee December 8, 2019 at 6:08 am #

      Perhaps. Try it and compare skill of the resulting model to a single model?

  17. Agamemnon December 19, 2019 at 11:20 pm #

    Thanks for yet another great post Jason!

    BTW, I am pretty sure that’s a Rotweiler, not a Doberman. I used a NASNetMobile model and it got it right.

    • Jason Brownlee December 20, 2019 at 6:49 am #

      Thanks for sharing! I don’t know about dogs πŸ™‚

  18. maalid January 16, 2020 at 5:47 am #

    Great Article Jason!

    I wondered if is posible to use my own pretrained keras model (with 3 output classes) for doing transfer learning. I mean, I have used VGG16 to train a new 3 output model and now I want to add another class, but instead adding four output to the VGG16 from the scratch I think maybe I can use the already 3 classes trained model.

    If so, can you give some lights how to do it?

    Thanks in advance!

    • Jason Brownlee January 16, 2020 at 6:28 am #


      Yes. Load the model as a feature extractor above and add a new output layer and only train the new output layer.

      I give examples here:

      • maalid January 16, 2020 at 7:41 am #

        Wow, thanks for your fast reply!

        Reading the example you gave me, I think I got it:

        1. Load the baseline model I already have (the VGG16+3_output_classes.h5 model)
        2. Mark loaded layers as not trainable
        3. Add new classifier layers with 4 outputs classes
        4. Define the new model (4 outputs classes)
        5. Compile new model
        6. Fit new model (having prepared the set of images)
        7. Enjoy πŸ™‚

        Seems pretty simple and easy to implement, definitely I will try it!

        Thanks again Jason!

  19. Rishang February 6, 2020 at 7:30 pm #

    model = VGG16(include_top=False, weights=’imagenet’, input_shape=(150,150,3))

    “model.outputs” is a list of tensors, a list of all the output tensors of the VGG16 model.

    Whereas ‘model.output’ is the single output tensor

    When I use model.outputs as input to my own layer:

    conv1 = Conv2D(32, kernel_size=4, activation=’relu’) (model.outputs)

    It gives me error-

    “int() argument must be a string, a bytes-like object or a number, not ‘TensorShape’ ”

    By using, model.output instead of model.outputs, I have solved the problem.

    Can you explain me why??
    And am I doing wrong by giving model.output instead of model.outputs ??

    • Jason Brownlee February 7, 2020 at 8:13 am #

      I would expect the output of the last later and the output of the model to be identical.

      Perhaps check the code?

  20. Umair March 10, 2020 at 5:51 am #


    I am trying to use VGG16 but facing a problem. I will thank full to you if you help me

    This is the code of architecture

    vgg_conv = VGG16(weights=’imagenet’, include_top=False, input_shape=(765, 415, 3))

    for layer in vgg_conv.layers[:-4]:
    layer.trainable = False

    for layer in vgg_conv.layers:
    print(layer, layer.trainable)

    model = models.Sequential()
    model.add(layers.Dense(128, activation=’relu’))


    When I call the fit method I face below error

    ValueError: Error when checking input: expected vgg16_input to have 4 dimensions, but got array with shape (559, 1, 760, 415, 3)

    I did search on it but could not find a proper solution, I am a beginner in this domain, please guide me accordingly.

    • Jason Brownlee March 10, 2020 at 6:36 am #

      Perhaps use the summary() function and use the output to debug the transforms of data through your model.

  21. mourad March 23, 2020 at 4:09 am #

    so, i want to pass these extracted features and pass them to an SVM classifier
    how to do that !

  22. Ritam Majumdar March 27, 2020 at 3:04 am #

    While trying the last part of the code in the article (fine tuning), I get the following error:
    InvalidArgumentError: Matrix size-incompatible: In[0]: [1,16384], In[1]: [512,10]
    [[{{node Final_5/BiasAdd}}]]

    My Code:
    # example of tending the vgg16 model
    from keras.applications.vgg16 import VGG16
    from keras.models import Model
    from keras.layers import Dense
    from keras.layers import Flatten
    # load model without classifier layers
    model = VGG16(include_top=False, input_shape=(32, 32, 3))
    # add new classifier layers
    flat1 = Flatten()(model.outputs)
    class1 = Dense(1024, activation=’relu’, name=’Pre-final’)(flat1)
    output = Dense(10, activation=’softmax’, name=’Final’)(class1)
    # define new model
    model = Model(inputs=model.inputs, outputs=output)
    # summarize

    for layer in model.layers:
    layer.trainable = False
    # mark some layers as not trainable

    model.get_layer(‘Pre-final’).trainable = True
    model.get_layer(‘Final’).trainable = True


    opt = keras.optimizers.RMSprop(0.0001, decay=1e-6)

    # Let’s train the model using RMSprop
    x_train = x_train.astype(‘float32’)
    x_test = x_test.astype(‘float32’)
    x_train /= 255
    x_test /= 255, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_test, y_test), shuffle=True)

    What am I missing?

  23. UDAYA BHANU SATAPATHY April 13, 2020 at 12:17 am #

    Wonderful article. Bookmarked!

  24. Sayanthan April 25, 2020 at 10:10 am #

    Hi Jason,

    I was having trouble in understanding the transfer learning using the tensorflow, but after reading the materials, it just became crystal clear.

    Really appreciate your work mate.

  25. Laique Merlin Djeutchouang April 26, 2020 at 7:15 am #

    Thanks again Jason for this amazing post. You always make it easy πŸ‘ŒπŸΎ

  26. Aldo Battista May 17, 2020 at 11:03 pm #

    Hey, Jason, thanks for the post, really interesting. I wanted to ask you something, by the way. So it’s very clear to me how to use pretrained models to do transfer learning but in the specific application I have in mind maybe it’s a bit complicated. I’d like to build a decoder that takes the high-level representations generated by pretrained networks, such as VGG, and reconstruct the starting images. I imagine that to do this you need a very complicated and difficult network to train so I was wondering if there will also be pretrained versions of the decoders associated with these pretrained networks. I hope I made myself clear, thank you for your possible answer.

  27. Ris May 19, 2020 at 9:55 pm #

    please tell me how can i get the mean of the feature vetors

  28. Ris May 20, 2020 at 6:34 am #

    while running loop for loading all the pickle files of feature vectors ,i am getting error that the the feature vector file not found.i am sure i gave the absolute path of the directory already.but while individually load a feature vector pickle file ,i am able to open it.plslet me know how can i load all my feature vectures using loop

  29. nkm May 23, 2020 at 11:34 pm #

    Hi Mr Jason,
    Please suggest use of transfer learning for grayscale images. How it can be modified for grayscale black & white images?

    • Jason Brownlee May 24, 2020 at 6:10 am #

      Define the input to take images with the preferred size and channels of your data.

      • nkm May 24, 2020 at 8:11 pm #

        Thanks for quick reply.

        It is to be noted that all transfer learning models (like, VGG, ReNet..) are developed for RGB, ie for 3 channels. However, I would like to use these models for grayscale images ie with no of channels =1. Please suggest modifications or how I can implement RGB based models for black & white case using transfer learning.

        Resizing is not an issue, it is configurable.

        • Jason Brownlee May 25, 2020 at 5:47 am #

          The above tutorial shows you how to define a new input shape, including channels, for the vgg model. Use that as a starting point and change it to 1 channel.

          • nkm May 28, 2020 at 4:52 am #

            Hello Mr Jason,

            As per your instruction, I changed no of channels to 1 then it gives following error:
            ValueError: The input must have 3 channels; got input_shape=(224, 224, 1)

            However, when I kept no of channels to 3 only, I get following error:
            InvalidArgumentError: input depth must be evenly divisible by filter depth: 1 vs 3
            [[node model_4/block1_conv1/Conv2D (defined at :22) ]] [Op:__inference_train_function_48474]

            Kindly help to resolve error.

          • Jason Brownlee May 28, 2020 at 6:21 am #

            Sorry, I do not have the capacity to debug this for you. You will have to develop a solution yourself.

  30. Pellumb May 24, 2020 at 6:35 am #

    Hello, i am trying to detect 10 different plant disease and i have small data. Do you suggest me to use transfer learning in this problem in order to increase the accuracy?

    Thanks in advance!

    • Jason Brownlee May 25, 2020 at 5:42 am #

      Yes, I would recommend using transfer learning and perhaps image data augmentation.

      • Pellumb May 25, 2020 at 10:34 am #

        Thanks for your response!

  31. Mereb Hayl June 12, 2020 at 4:05 pm #

    Hello Dear Jason, Thanks for the great lesson!

    I was trying to apply transfer learning to an “ssd_mobilenet_v1_coco_2017” pretrained object detection model. and was striving to modify the big model by reducing layers, and filters and couldn’t.
    Though I tried hard, I couldn’t even reduce the 3 color channel to one.
    Could you please suggest me a way to do this?

    Thanks ! !

    • Jason Brownlee June 13, 2020 at 5:48 am #

      Sorry to hear that, I don’t know what the problem could be off the cuff. Perhaps post your code and error message to stackoverflow.

  32. Zeke June 14, 2020 at 1:05 am #

    Hi Dr Jason!
    Thanks a lot for the great article!

    I am trying to apply transfer learning, but instead of using VGG16, I want to use a previously trained model with CIFAR10 that I saved as a h5 file. In precise, I want to use transfer learning for the model to be able to classify vehicle types (20 classes) with my datasets on different vehicles.

    May I know how do I use the h5 file as a pretrained model? I tried searching around but to no avail, as examples are using Keras’s pretrained models such as VGG16 from keras.applications. I was thinking that the problems that I have encountered is due to the different output sizes at the Dense. I tried model.pop for this but it does not seem to work either. It would be very helpful if you could provide some pointers on this, thank you!

    • Jason Brownlee June 14, 2020 at 6:35 am #

      Sounds great.

      Yes, load the model, replace the input and/or output layers as needed with new layers, fine tune these new layers on your new dataset and away you go!

      • Zeke June 16, 2020 at 11:41 am #

        I shall give it a go. Thank you so much!

  33. JAVAID AHMAD WANI June 23, 2020 at 10:06 pm #

    plz help me my dataset contains 38 classes
    the dataset i am using is ( new plant diseases dataset)

    train_path = Path(‘C:\\Users\\User\AnacondaProjects\\Plant Disease Detection and Classification ( Final year Project )\\New Plant Diseases Dataset\\train’)
    test_path = Path(‘C:\\Users\\User\AnacondaProjects\\Plant Disease Detection and Classification ( Final year Project )\\New Plant Diseases Dataset\\test’)
    valid_path = Path(‘C:\\Users\\User\AnacondaProjects\\Plant Disease Detection and Classification ( Final year Project )\\New Plant Diseases Dataset\\valid’)

    train_batches = ImageDataGenerator().flow_from_directory(train_path, target_size=(224,224), classes=[‘Apple___Apple_scab’,’Apple___Black_rot’, ‘Apple___Cedar_apple_rust’, ‘Apple___healthy’, ‘Blueberry___healthy’, ‘Cherry_(including_sour)___healthy’, ‘Cherry_(including_sour)___Powdery_mildew’, ‘Corn_(maize)___Cercospora_leaf_spot Gray_leaf_spot’, ‘Corn_(maize)___Common_rust_’, ‘Corn_(maize)___healthy’, ‘Corn_(maize)___Northern_Leaf_Blight’, ‘Grape___Black_rot’, ‘Grape___Esca_(Black_Measles)’, ‘Grape___healthy’, ‘Grape___Leaf_blight_(Isariopsis_Leaf_Spot)’, ‘Orange___Haunglongbing_(Citrus_greening)’,’Peach___Bacterial_spot’, ‘Peach___healthy’, ‘Pepper,_bell___Bacterial_spot’, ‘Pepper,_bell___healthy’, ‘Potato___Early_blight’, ‘Potato___healthy’, ‘Potato___Late_blight’, ‘Raspberry___healthy’, ‘Soybean___healthy’, ‘Squash___Powdery_mildew’, ‘Strawberry___healthy’, ‘Strawberry___Leaf_scorch’, ‘Tomato___Bacterial_spot’, ‘Tomato___Early_blight’, ‘Tomato___healthy’, ‘Tomato___Late_blight’, ‘Tomato___Leaf_Mold’, ‘Tomato___Septoria_leaf_spot’, ‘Tomato___Spider_mites Two-spotted_spider_mite’, ‘Tomato___Target_Spot’, ‘Tomato___Tomato_mosaic_virus’, ‘Tomato___Tomato_Yellow_Leaf_Curl_Virus’], batch_size=10)
    test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size=(224,224), classes=[‘Apple___Apple_scab’,’Apple___Black_rot’, ‘Apple___Cedar_apple_rust’, ‘Apple___healthy’, ‘Blueberry___healthy’, ‘Cherry_(including_sour)___healthy’, ‘Cherry_(including_sour)___Powdery_mildew’, ‘Corn_(maize)___Cercospora_leaf_spot Gray_leaf_spot’, ‘Corn_(maize)___Common_rust_’, ‘Corn_(maize)___healthy’, ‘Corn_(maize)___Northern_Leaf_Blight’, ‘Grape___Black_rot’, ‘Grape___Esca_(Black_Measles)’, ‘Grape___healthy’, ‘Grape___Leaf_blight_(Isariopsis_Leaf_Spot)’, ‘Orange___Haunglongbing_(Citrus_greening)’,’Peach___Bacterial_spot’, ‘Peach___healthy’, ‘Pepper,_bell___Bacterial_spot’, ‘Pepper,_bell___healthy’, ‘Potato___Early_blight’, ‘Potato___healthy’, ‘Potato___Late_blight’, ‘Raspberry___healthy’, ‘Soybean___healthy’, ‘Squash___Powdery_mildew’, ‘Strawberry___healthy’, ‘Strawberry___Leaf_scorch’, ‘Tomato___Bacterial_spot’, ‘Tomato___Early_blight’, ‘Tomato___healthy’, ‘Tomato___Late_blight’, ‘Tomato___Leaf_Mold’, ‘Tomato___Septoria_leaf_spot’, ‘Tomato___Spider_mites Two-spotted_spider_mite’, ‘Tomato___Target_Spot’, ‘Tomato___Tomato_mosaic_virus’, ‘Tomato___Tomato_Yellow_Leaf_Curl_Virus’], batch_size=40)
    valid_batches = ImageDataGenerator().flow_from_directory(valid_path, target_size=(224,224), classes=[‘Apple___Apple_scab’,’Apple___Black_rot’, ‘Apple___Cedar_apple_rust’, ‘Apple___healthy’, ‘Blueberry___healthy’, ‘Cherry_(including_sour)___healthy’, ‘Cherry_(including_sour)___Powdery_mildew’, ‘Corn_(maize)___Cercospora_leaf_spot Gray_leaf_spot’, ‘Corn_(maize)___Common_rust_’, ‘Corn_(maize)___healthy’, ‘Corn_(maize)___Northern_Leaf_Blight’, ‘Grape___Black_rot’, ‘Grape___Esca_(Black_Measles)’, ‘Grape___healthy’, ‘Grape___Leaf_blight_(Isariopsis_Leaf_Spot)’, ‘Orange___Haunglongbing_(Citrus_greening)’,’Peach___Bacterial_spot’, ‘Peach___healthy’, ‘Pepper,_bell___Bacterial_spot’, ‘Pepper,_bell___healthy’, ‘Potato___Early_blight’, ‘Potato___healthy’, ‘Potato___Late_blight’, ‘Raspberry___healthy’, ‘Soybean___healthy’, ‘Squash___Powdery_mildew’, ‘Strawberry___healthy’, ‘Strawberry___Leaf_scorch’, ‘Tomato___Bacterial_spot’, ‘Tomato___Early_blight’, ‘Tomato___healthy’, ‘Tomato___Late_blight’, ‘Tomato___Leaf_Mold’, ‘Tomato___Septoria_leaf_spot’, ‘Tomato___Spider_mites Two-spotted_spider_mite’, ‘Tomato___Target_Spot’, ‘Tomato___Tomato_mosaic_virus’, ‘Tomato___Tomato_Yellow_Leaf_Curl_Virus’], batch_size=10)

    import keras
    vgg16_model = keras.applications.vgg16.VGG16(include_top= True, weights=’imagenet’, input_tensor=None, input_shape=None, pooling=None, classes=38)

    vgg16_model = Sequential()
    for layer in model.layers[:-1]:
    print(‘ model copied’)

    for layer in vgg16_model.layers:
    layer.trainable = False

    vgg16_model.add(Dense(38, activation = ‘softmax’))
    print(‘add last layer’)


    train_num = train_batches.samples
    valid_num = valid_batches.samples

    # checkpoint
    from keras.callbacks import ModelCheckpoint
    weightpath = “best_weights_9.hdf5”
    checkpoint = ModelCheckpoint(weightpath, monitor=’val_accuracy ‘, verbose=1, save_best_only=True, save_weights_only=True, mode=’max’)
    callbacks_list = [checkpoint]

    while executing below lines tt shows me an error which is displayed at last

    model.fit_generator(train_batches, steps_per_epoch=train_num//batch_size,
    validation_data = valid_batches, validation_steps=valid_num//batch_size,
    epochs = 25 ,callbacks=callbacks_list)

    while executing

  34. Mosharof June 28, 2020 at 10:30 pm #


  35. Mohamad July 6, 2020 at 9:12 pm #

    Hi Jason,
    This post is awesome. Thanks a lot for the details.

    I have a question please. You say: “We can then use the Keras function API to add a new Flatten layer after the last pooling layer in the VGG16 model, then define a new classifier model with a Dense fully connected layer and an output layer that will predict the probability for 10 classes”

    Why did we add one flatten and one dense layers? We removed the last Dense layer of the model. So, why am I adding a flatten one as well?


    • Jason Brownlee July 7, 2020 at 6:36 am #


      The output of the pooling layer is a bunch of 2d filter maps, we need to flatten them to a vector to pass into a dense layer for classification.

Leave a Reply