Save and Load Your Keras Deep Learning Models

Keras is a simple and powerful Python library for deep learning.

Given that deep learning models can take hours, days and even weeks to train, it is important to know how to save and load them from disk.

In this post, you will discover how you can save your Keras models to file and load them up again to make predictions.

Let’s get started.

  • Update Mar 2017: Added instructions to install h5py first. Added missing brackets on final print statement in each example.
Save and Load Your Keras Deep Learning Models

Save and Load Your Keras Deep Learning Models
Photo by art_inthecity, some rights reserved.

Tutorial Overview

Keras separates the concerns of saving your model architecture and saving your model weights.

Model weights are saved to HDF5 format. This is a grid format that is ideal for storing multi-dimensional arrays of numbers.

The model structure can be described and saved using two different formats: JSON and YAML.

In this post we are going to look at two examples of saving and loading your model to file:

  • Save Model to JSON.
  • Save Model to YAML.

Each example will also demonstrate saving and loading your model weights to HDF5 formatted files.

The examples will use the same simple network trained on the Pima Indians onset of diabetes binary classification dataset. This is a small dataset that contains all numerical data and is easy to work with. You can download this dataset and place it in your working directory with the filename “pima-indians-diabetes.csv“.

Confirm that you have the latest version of Keras installed (v1.2.2 as of March 2017).

Note: You may need to install h5py first:

Beat the Math/Theory Doldrums and Start using Deep Learning in your own projects Today, without getting lost in “documentation hell”

Deep Learning With Python Mini-CourseGet my free Deep Learning With Python mini course and develop your own deep nets by the time you’ve finished the first PDF with just a few lines of Python.

Daily lessons in your inbox for 14 days, and a DL-With-Python “Cheat Sheet” you can download right now.   

Download Your FREE Mini-Course  

 

Save Your Neural Network Model to JSON

JSON is a simple file format for describing data hierarchically.

Keras provides the ability to describe any model using JSON format with a to_json() function. This can be saved to file and later loaded via the model_from_json() function that will create a new model from the JSON specification.

The weights are saved directly from the model using the save_weights() function and later loaded using the symmetrical load_weights() function.

The example below trains and evaluates a simple model on the Pima Indians dataset. The model is then converted to JSON format and written to model.json in the local directory. The network weights are written to model.h5 in the local directory.

The model and weight data is loaded from the saved files and a new model is created. It is important to compile the loaded model before it is used. This is so that predictions made using the model can use the appropriate efficient computation from the Keras backend.

The model is evaluated in the same way printing the same evaluation score.

Running this example provides the output below.

The JSON format of the model looks like the following:

Save Your Neural Network Model to YAML

This example is much the same as the above JSON example, except the YAML format is used for the model specification.

The model is described using YAML, saved to file model.yaml and later loaded into a new model via the model_from_yaml() function. Weights are handled in the same way as above in HDF5 format as model.h5.

Running the example displays the following output:

The model described in YAML format looks like the following:

Further Reading

Summary

In this post, you discovered how to serialize your Keras deep learning models.

You learned how you can save your trained models to files and later load them up and use them to make predictions.

You also learned that model weights are easily stored using  HDF5 format and that the network structure can be saved in either JSON or YAML format.

Do you have any questions about saving your deep learning models or about this post? Ask your questions in the comments and I will do my best to answer them.

Frustrated With Your Progress In Deep Learning?

 What If You Could Develop Your Own Deep Nets in Minutes

...with just a few lines of Python

Discover how in my new Ebook: Deep Learning With Python

It covers self-study tutorials and end-to-end projects on topics like:
Multilayer PerceptronsConvolutional Nets and Recurrent Neural Nets, and more...

Finally Bring Deep Learning To
Your Own Projects

Skip the Academics. Just Results.

Click to learn more.

28 Responses to Save and Load Your Keras Deep Learning Models

  1. Onkar August 31, 2016 at 4:21 pm #

    Hi Jason,

    I am grateful you for sharing knowledge through this blog. It has been very helpful for me.
    Thank you for the effort.

    I have one question. When I am executing keras code to load YAML / JSON data i am seeing following error.

    Traceback (most recent call last):
    File “simple_rnn.py”, line 158, in
    loaded_model = model_from_yaml(loaded_model_yaml)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/models.py”, line 26, in model_from_yaml
    return layer_from_config(config, custom_objects=custom_objects)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/utils/layer_utils.py”, line 35, in layer_from_config
    return layer_class.from_config(config[‘config’])
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/models.py”, line 781, in from_config
    layer = get_or_create_layer(first_layer)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/models.py”, line 765, in get_or_create_layer
    layer = layer_from_config(layer_data)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/utils/layer_utils.py”, line 35, in layer_from_config
    return layer_class.from_config(config[‘config’])
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/engine/topology.py”, line 896, in from_config
    return cls(**config)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/layers/recurrent.py”, line 290, in __init__
    self.init = initializations.get(init)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/initializations.py”, line 109, in get
    ‘initialization’, kwargs=kwargs)
    File “/usr/local/lib/python2.7/dist-packages/Keras-1.0.4-py2.7.egg/keras/utils/generic_utils.py”, line 14, in get_from_module
    str(identifier))
    Exception: Invalid initialization:

    What could be the reason ? File is getting saved properly but at the time of loading model I am facing this issue.
    Can you please give me any pointers ?

    Thanks,
    Onkar

    • Jason Brownlee September 1, 2016 at 7:57 am #

      Sorry Onkar, the fault is not clear.

      Are you able to execute the example in the tutorial OK?

  2. Walid Ahmed September 27, 2016 at 1:54 am #

    your code worked fine,
    I tried to add saving model to my code but the files were not actually created alyhough I got no error messages

    please advice

    walid

    • Jason Brownlee September 27, 2016 at 7:45 am #

      I expect the files were created. Check your current working directory / the dir where the source code files are located.

  3. Peter September 27, 2016 at 1:43 pm #

    Hi Jason,

    Thanks for creating this valuable content.

    On my Mac (OSX10.11), the script ran fine until the last line, in which it gave a syntax error below:

    >>> print “%s: %.2f%%” % (loaded_model.metrics_names[1], score[1]*100)
    File “”, line 1
    print “%s: %.2f%%” % (loaded_model.metrics_names[1], score[1]*100)
    ^
    SyntaxError: invalid syntax

    What could be the issue here?

    Thanks,
    Peter

    • Jason Brownlee September 28, 2016 at 7:37 am #

      Hi Peter, you may be on Python3, try adding brackets around the argument to the print functions.

  4. Deployment September 29, 2016 at 1:03 am #

    Hi,

    Your blog and books were great, and thanks much to you I finally got my project working in Keras.

    I can’t seem to find how to translate a Keras model in to a standalone code that can run without Keras installed.

    The best I could find was to learn TensorFlow, build an equivalent model in TF, then use TF to create standalone code.

    Does Keras not have such functionality?

    Thanks

    • Jason Brownlee September 29, 2016 at 8:37 am #

      Hi, my understanding is that Keras is required to use the model in prediction.

      You could try to save the network weights and use them in your own code, but you are creating a lot of work for yourself.

  5. Davood November 22, 2016 at 10:48 am #

    Hello Jason,

    Thanks for your great and very helpful website.

    Since in here you talked about how to save a model, I wanted to know how we can save an embedding layer in the way that can be seen in a regular word embeddings file (i.e. text file or txt format). Let’s assume we either learn these word embeddings in the model from scratch or we update those pre-trained ones which are fed in the first layer of the model.

    I truly appreciate your response in advance.

    Regards,
    Davood

    • Jason Brownlee November 23, 2016 at 8:49 am #

      I’m not sure we need to save embedding layers Davood.

      I believe they are deterministic and can just be re-created.

      • Davood November 23, 2016 at 10:51 am #

        I guess we should be able to save word embeddings at one point (not needed always though!). To visualize/map them in a (2D) space or to test algebraic word analogies on them can be some examples of this need.

        I found the answer for this and I’m sharing this here:

        If we train an embedding layer emb (e.g. emb = Embedding(some_parameters_here) ), we can get the resulting word-by-dimension matrix by my_embeddings = emb.get_weights(). Then, we can do normal numpy things like np.save(“my_embeddings.npy”, my_matrix) to save this matrix; or use other built-in write_to_a_file functions in Python to store each line of this matrix along with its associated word. These words and their indices are typically stored in a word_index dictionary somewhere in the code.

        • Jason Brownlee November 24, 2016 at 10:36 am #

          Very nice, thanks for sharing the specifics Davood.

          • Davood November 29, 2016 at 7:25 pm #

            You are very welcome Jason.
            However I have another question here!
            Let’s assume we two columns of networks in keras and these two columns are exactly the same. These two are going to merge on the top and then feed into a dense layer which is the output layer in our model. My question is, while the first layer of each column here is an embedding layer, how can we share the the weights of the similar layers in the columns? No need to say that we set our embedding layers (first layers) in a way that we only have one embedding matrix. What I mean is shared embeddings, something like this:
            emb1 = Embedding(some_parameters_here)
            emb2 = emb1 # instead of emb2 = Embedding(some_other_parameters_here)).
            How about the other layers on top of these two embedding layers?! How to share their weights?
            Thanks for your answer in advance.

          • Jason Brownlee November 30, 2016 at 7:55 am #

            Hmm, interesting Davood.

            I think, and could be wrong, that embedding layers are deterministic. They do not have state, only the weights in or out have state. Create two and use them side by side. Try it and see.

            I’d love to know how you go?

  6. Chao December 6, 2016 at 1:58 am #

    Hi Jason, thanks for your share, it helps me a lot. I’d like to ask a question, why the optimizer while compiling the model is adam, but uses rmsprop instead while compiling the loaded_model?

    • Jason Brownlee December 6, 2016 at 9:53 am #

      I would suggest trying many different optimizers and see what you like best / works best for your problem.

      I find ADAM is fast and gives good results.

  7. kl December 13, 2016 at 11:47 pm #

    I have difficulties finding an answer to this question:

    when are weights initialized in keras?

    at compile time ? (probably not)

    on first epoch ?

    This is important when resuming learning

    • Jason Brownlee December 14, 2016 at 8:28 am #

      Interesting question.

      I don’t know.

      If I had to guess, I would say at the model.compile() time when the data structures are created.

      It might be worth asking on the keras email list – I’d love to know the answer.

  8. Soheil December 22, 2016 at 8:44 pm #

    Thank you for creating such great blog.
    I saved a model with mentioned code. But when I wanted to load it again, I faced following error. It seems the network architecture was not save correctly?

    —————————————————————————
    Exception Traceback (most recent call last)
    in ()
    1 # load weights into new model
    —-> 2 modelN.load_weights(“model41.h5”)
    3 print(“Loaded model from disk”)

    C:\Anaconda2\envs\py35\lib\site-packages\keras\engine\topology.py in load_weights(self, filepath, by_name)
    2518 self.load_weights_from_hdf5_group_by_name(f)
    2519 else:
    -> 2520 self.load_weights_from_hdf5_group(f)
    2521
    2522 if hasattr(f, ‘close’):

    C:\Anaconda2\envs\py35\lib\site-packages\keras\engine\topology.py in load_weights_from_hdf5_group(self, f)
    2570 ‘containing ‘ + str(len(layer_names)) +
    2571 ‘ layers into a model with ‘ +
    -> 2572 str(len(flattened_layers)) + ‘ layers.’)
    2573
    2574 # We batch weight value assignments in a single backend call

    Exception: You are trying to load a weight file containing 4 layers into a model with 5 layers.

    • Jason Brownlee December 23, 2016 at 5:31 am #

      Hi Soheil,

      It looks like the network structure that you are loading the weights into does not match the structure of the weights.

      Double check that the network structure matches exactly the structure that you used when you saved the weights. You can even save this structure as a json or yaml file as well.

  9. prajnya January 24, 2017 at 6:22 pm #

    Hi Jason,

    I have a question. Now that I have saved the model and the weights, is it possible for me to come back after a few days and train the model again with initial weights equal to the one I saved?

    • Jason Brownlee January 25, 2017 at 9:58 am #

      Great question prajnya.

      You can load the saved weights and continue training/update with new data or start making predictions.

  10. AKSHAY February 8, 2017 at 6:06 pm #

    Hi Jason,

    It is an amazing blog you have here. Thanks for the well documented works.
    I have a question regarding loading the model weights. Is there a way to save the weights into a variable rather than loading and assigning the weights to a different model?
    I wanted to do some operations on the weights associated with the intermediate hidden layer.

    I was anticipating on using ModelCheckpoint but I am a bit lost on reading weights from the hdf5 format and saving it to a variable. Could you please help me figure it out.

    Thanks

    • Jason Brownlee February 9, 2017 at 7:24 am #

      Great question, sorry I have not done this.

      I expect you will be able to extract them using the Keras API, it might be worth looking at the source code on github.

  11. Patrick March 1, 2017 at 3:53 am #

    Hi Jason

    thanks a lot for your excellent tutorials! Very much appreciated…

    Regarding the saving and loading: It seems that Keras as of now saves model and weights in HD5 rather than only the weights.

    https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model

    This results in a much simpler snippet for import / export:

    ——————————————————-

    from keras.models import load_model

    model.save(‘my_model.h5’) # creates a HDF5 file ‘my_model.h5’
    del model # deletes the existing model

    # returns a compiled model
    # identical to the previous one
    model = load_model(‘my_model.h5’)

    ——————————————————-

    • Jason Brownlee March 1, 2017 at 8:42 am #

      Thanks Patrick, I’ll investigate and look at updating the post soon.

  12. Avik Moulik March 7, 2017 at 8:11 am #

    Getting this error:

    NameError: name ‘model_from_json’ is not defined

    Thanks in advance for any help on this.

    • Jason Brownlee March 7, 2017 at 9:38 am #

      Confirm that you have Keras 1.2.2 or higher installed.

Leave a Reply