Two of the top numerical platforms in Python that provide the basis for Deep Learning research and development are Theano and TensorFlow.
Both are very powerful libraries, but both can be difficult to use directly for creating deep learning models.
In this post, you will discover the Keras Python library that provides a clean and convenient way to create a range of deep learning models on top of Theano or TensorFlow.
Kick-start your project with my new book Deep Learning With Python, including step-by-step tutorials and the Python source code files for all examples.
Let’s get started.
Update Oct/2016: Updated for Keras 1.1.0, Theano 0.8.2 and TensorFlow 0.10.0.
Update Sep/2019: Updated for Keras 2.2.5 API.
Introduction to the Python Deep Learning Library Keras Photo by Dennis Jarvis, some rights reserved.
What is Keras?
Keras is a minimalist Python library for deep learning that can run on top of Theano or TensorFlow.
It was developed to make implementing deep learning models as fast and easy as possible for research and development.
It runs on Python 2.7 or 3.5 and can seamlessly execute on GPUs and CPUs given the underlying frameworks. It is released under the permissive MIT license.
Keras was developed and maintained by François Chollet, a Google engineer using four guiding principles:
Modularity: A model can be understood as a sequence or a graph alone. All the concerns of a deep learning model are discrete components that can be combined in arbitrary ways.
Minimalism: The library provides just enough to achieve an outcome, no frills and maximizing readability.
Extensibility: New components are intentionally easy to add and use within the framework, intended for researchers to trial and explore new ideas.
Python: No separate model files with custom file formats. Everything is native Python.
How to Install Keras
Keras is relatively straightforward to install if you already have a working Python and SciPy environment.
You must also have an installation of Theano or TensorFlow on your system already.
You can see installation instructions for both platforms here:
Keras can be installed easily using PyPI, as follows:
1
sudo pip install keras
At the time of writing, the most recent version of Keras is version 2.2.5. You can check your version of Keras on the command line using the following snippet:
You can check your version of Keras on the command line using the following snippet:
1
python -c "import keras; print keras.__version__"
Running the above script you will see:
1
2.2.5
You can upgrade your installation of Keras using the same method:
1
sudo pip install --upgrade keras
Theano and TensorFlow Backends for Keras
Assuming you have both Theano and TensorFlow installed, you can configure the backend used by Keras.
The easiest way is by adding or editing the Keras configuration file in your home directory:
1
~/.keras/keras.json
Which has the format:
1
2
3
4
5
6
{
"image_data_format": "channels_last",
"backend": "tensorflow",
"epsilon": 1e-07,
"floatx": "float32"
}
In this configuration file you can change the “backend” property from “tensorflow” (the default) to “theano“. Keras will then use the configuration the next time it is run.
You can confirm the backend used by Keras using the following snippet on the command line:
1
python -c "from keras import backend; print(backend.backend())"
Running this with default configuration you will see:
1
2
Using TensorFlow backend.
tensorflow
You can also specify the backend to use by Keras on the command line by specifying the KERAS_BACKEND environment variable, as follows:
1
KERAS_BACKEND=theano python -c "from keras import backend; print(backend.backend())"
Running this example prints:
1
2
Using Theano backend.
theano
Need help with Deep Learning in Python?
Take my free 2-week email course and discover MLPs, CNNs and LSTMs (with code).
Click to sign-up now and also get a free PDF Ebook version of the course.
Build Deep Learning Models with Keras
The focus of Keras is the idea of a model.
The main type of model is called a Sequence which is a linear stack of layers.
You create a sequence and add layers to it in the order that you wish for the computation to be performed.
Once defined, you compile the model which makes use of the underlying framework to optimize the computation to be performed by your model. In this you can specify the loss function and the optimizer to be used.
Once compiled, the model must be fit to data. This can be done one batch of data at a time or by firing off the entire model training regime. This is where all the compute happens.
Once trained, you can use your model to make predictions on new data.
We can summarize the construction of deep learning models in Keras as follows:
Define your model. Create a sequence and add layers.
Compile your model. Specify loss functions and optimizers.
Fit your model. Execute the model using data.
Make predictions. Use the model to generate predictions on new data.
You can develop your first deep learning neural network in Keras with just a few lines of code. See this step-by-step Keras Tutorial:
In this post, you discovered the Keras Python library for deep learning research and development.
You discovered that Keras is designed for minimalism and modularity allowing you to very quickly define deep learning models and run them on top of a Theano or TensorFlow backend.
Do you have any questions about Keras or this post? Ask your questions in the comments and I will do my best to answer them.
Great post, Jason! I will start my path in deep learning so I was wondering if you could help me with the hardware requirements. What are the minimal computational resources (disk space for saving/loading CNN models, RAM, GPU time) needed to run keras/theano/tensor flow?
Larger networks like big CNNs do need big hardware, like GPUs. I recommend using GPUs on AWS, it can be very cost effective, e.g. less than $1 an hour.
Generally, when you use Theano as the backend and it is configured to use the GPU (by flags at runtime or in the config), it will report that it is using the GPU on the command line.
From the site:
If you want to find whether Keras backend uses a GPU or CPU:
1
2
3
4
5
6
7
8
9
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
[name:"/device:CPU:0"
device_type:"CPU"
memory_limit:268435456
locality{
}
incarnation:1807102777791646816
]
So my particular version of Keras uses the CPU for the particular backend.
My particular version of Keras has the Tensorflow backend. So my Tensorflow installation uses the CPU.
To set the Tensorflow for either GPU or CPU processing is not so straightforward.
I stand corrected on this, but It appears that you cannot programmatically switch between the GPU and CPU version of Tensorflow.
I shall not discus what particular brand of GPU to use because “pipping” the GPU version of Tensorflow appears to be brand agnostic, eg Nvidia’s CUDA is one brand..
Nevertheless, if you want to install the GPU or CPU versions of Tensorflow, do this:
Dear Dr Jason and Marcel,
The reason why I chose Tensorflow over Theano is that the latest release of Theano in January 2019 was a maintenance release with bug fixes but no new features, reference https://pypi.org/project/Theano/.
In addition, if you look at the github repository at https://github.com/Theano/Theano particularly the HISTORY.txt file, all the Theano releases have been maintenance with no additional features. In addition most of the folders have not been updated for a long time except the Theano folder (1 month ago as at 12/10/2019 and the “update the documentation” as at 9 months ago as at 12/10/2019).
Wanted to ask – maybe you have tried Keras for 3D convolution? I ran into some questions with it and cannot get help from the Keras team (https://github.com/fchollet/keras/issues/3561). Would be great if you know what is correct there!
My students on windows use Theano. In fact, generally, I recommend Theano over tensorflow in my books and tutorials. At least for now, it is faster on general hardware.
I would like to know if there is a way to prepare a dataset which going to be string sequence of DNA “ATGCCGATGA along with its label ? i searched everywhere couldn’t find any answer.
There’s a new book called ‘Elegant Scipy’ that has examples of using numpy to normalize gene expression data that encodes letters into numbers using that might be useful.
Hi Jason!
Each time I run the Keras, I get inconsistent result.
Is there any way that it converges to the same solution as we have ‘random_state’ in sklearn which helps us getting the same solution how many ever times we run it.
Please help me to get out of this issue.
Thanks for this very useful post. I was convinced! I am moving from pyBrain to Keras now.
I found the format and structure of your blog very good. I am initiating my own to help in an attempt to provide and additional source for my students.
I have not decided the tools yet, but I would like to make post directly form a jupyter notebook. Is this your case? If so, could you share your workflow and tools used, please?
I found specially interesting the applet you used to show code blocks, which allow toggle the line number on and off.
what is the actual difference between neural network learning and deep neural network(deep learning) learning.?? is it only in weight initialization in each layer? or in back propagation algorithm??
thanks in advance
I am working on a small project where I need to detect the emotion using 3d camera (from microsoft). I will be using CNN for this..but is it possible to use keras in this case.
It would be in real time.
Always on the cutting edge of machine learning!
Thanks
Thanks Victor.
Great post, Jason! I will start my path in deep learning so I was wondering if you could help me with the hardware requirements. What are the minimal computational resources (disk space for saving/loading CNN models, RAM, GPU time) needed to run keras/theano/tensor flow?
You can run small networks on your CPU.
Larger networks like big CNNs do need big hardware, like GPUs. I recommend using GPUs on AWS, it can be very cost effective, e.g. less than $1 an hour.
Hi Jason, how do I check that my neural networks are trained using my GPU, and not only my CPU?
Great question.
Generally, when you use Theano as the backend and it is configured to use the GPU (by flags at runtime or in the config), it will report that it is using the GPU on the command line.
Dear Dr Jason and Marcel,
I would like to add that if you are using Tensorflow as the back end, there are two installations versions: GPU or CPU.
My reference is this Stackexchange site https://stackoverflow.com/questions/45068243/how-to-check-if-keras-tensorflow-backend-is-gpu-or-cpu-version.
From the site:
If you want to find whether Keras backend uses a GPU or CPU:
So my particular version of Keras uses the CPU for the particular backend.
My particular version of Keras has the Tensorflow backend. So my Tensorflow installation uses the CPU.
To set the Tensorflow for either GPU or CPU processing is not so straightforward.
I stand corrected on this, but It appears that you cannot programmatically switch between the GPU and CPU version of Tensorflow.
I shall not discus what particular brand of GPU to use because “pipping” the GPU version of Tensorflow appears to be brand agnostic, eg Nvidia’s CUDA is one brand..
Nevertheless, if you want to install the GPU or CPU versions of Tensorflow, do this:
GPU version of Tensorblow:
CPU version of Tensorflow:
Thank you,
Anthony of Sydney
Dear Dr Jason and Marcel,
The reason why I chose Tensorflow over Theano is that the latest release of Theano in January 2019 was a maintenance release with bug fixes but no new features, reference https://pypi.org/project/Theano/.
In addition, if you look at the github repository at https://github.com/Theano/Theano particularly the HISTORY.txt file, all the Theano releases have been maintenance with no additional features. In addition most of the folders have not been updated for a long time except the Theano folder (1 month ago as at 12/10/2019 and the “update the documentation” as at 9 months ago as at 12/10/2019).
Thank you,
Anthony of Sydney
It’s a good reason.
I still recommend Theano for any windows users having trouble installing TensorFlow. It works great for almost everything you need to do with keras.
Thanks for sharing.
Hi Jason!
Wanted to ask – maybe you have tried Keras for 3D convolution? I ran into some questions with it and cannot get help from the Keras team (https://github.com/fchollet/keras/issues/3561). Would be great if you know what is correct there!
Sorry Linara, I have not used Keras for a 3D CNN yet.
Ok, thank you for answer anyway!
Hi Jason,
It doesn’t appear that Tensorflow has a Windows install. Is there an alternative? If not, have you any insight if a windows version is in the works?
Many Thanks,
Alan
Hi Alan,
My students on windows use Theano. In fact, generally, I recommend Theano over tensorflow in my books and tutorials. At least for now, it is faster on general hardware.
I use Vagrant to set up a VM with Keras, TensorFlow and Theano installed. I run it on Windows.
https://github.com/nirmalyaghosh/deep-learning-vm if anyone is interested.
Great approach, thanks for sharing Nirmalya.
@Alan Macshane
U can get Tensorflow on Windows by getting Windows 10 insider preview build.
I am using Tensorflow on Windows
Hi Jason
Do u have any tutorial on classifying Gender from an image using Deep learning or classical Computer Vision approach?
It will be great help
I don’t Ranjeet, but that is a very cool project idea!
Hi Jason,
I would like to know if there is a way to prepare a dataset which going to be string sequence of DNA “ATGCCGATGA along with its label ? i searched everywhere couldn’t find any answer.
Thanks a lot in advance,
Hi Steven, consider encoding the letters as numbers and treat it as a sequence classification problem. Not dissimilar from sentiment analysis.
I hope that sparks some ideas.
There’s a new book called ‘Elegant Scipy’ that has examples of using numpy to normalize gene expression data that encodes letters into numbers using that might be useful.
Hi Jason,
YOU are wonderful.
Thanks Ali.
Can you guide me in installing TensorFlow for 32-bit linux?
Sorry Ash, I do not have a specific guide for TensorFlow on 32-bit Linux.
Any alternatives you’d suggest for using keras on 32-bit linux?
You may be able to run on 32-bit linux, I don’t know, sorry.
An alternative that will work is using amazon web services to run your models:
https://machinelearningmastery.com/develop-evaluate-large-deep-learning-models-keras-amazon-web-services/
I hope that helps.
Simple! Changing the backend to “theano” will allow you to run keras on 32-bit linux. Thanks anyways! Will look into AWS as well.
I’m glad to hear it Ash.
Hi Jason,
Great post
Do u have any tutorial on classification from a speech using recurrent neural network?
It will be great help
Not at this stage leila.
Good afternoon. I have lettle problem: I installed Keras, but “Sequential – no module named”. How do I this module connect?
What is the error exactly Nastya?
Are you able to confirm that the Keras library was installed successfully?
Do you know whether Microsoft is using Keras combined with Theano or TensorFlow in their Azure Machine Learning product (https://azure.microsoft.com/en-us/services/machine-learning/ ) ? Possibly some clue to this here however: https://blogs.technet.microsoft.com/machinelearning/2016/11/28/deep-learning-made-easy-in-azure/ but I am not sure..
Sorry, I don’t know.
is Keras compatible with python 3.6 ?
No, not according to the docs:
https://keras.io/
Try it anyway and see.
I am trying to specify theano as default backend using the below command in the terminal –
KERAS_BACKEND=theano python -c “from keras import backend; print(backend._BACKEND)”
It is giving the below error –
‘KERAS_BACKEND’ is not recognized as an internal or external command,
operable program or batch file.
Please let me know the solution.
Thanks & Regards,
Gopal
I’m sorry to hear that, perhaps it is specific to your platform – are you on windows?
Try changing the Keras configuration file instead, in ~/.keras/keras.json
Hi Jason!
Each time I run the Keras, I get inconsistent result.
Is there any way that it converges to the same solution as we have ‘random_state’ in sklearn which helps us getting the same solution how many ever times we run it.
Please help me to get out of this issue.
Thanks.
Yes, neural networks are stochastic.
I would recommend repeating your experiments multiple times and reporting the mean model performance.
Hi Jason,
Thanks for this very useful post. I was convinced! I am moving from pyBrain to Keras now.
I found the format and structure of your blog very good. I am initiating my own to help in an attempt to provide and additional source for my students.
I have not decided the tools yet, but I would like to make post directly form a jupyter notebook. Is this your case? If so, could you share your workflow and tools used, please?
I found specially interesting the applet you used to show code blocks, which allow toggle the line number on and off.
Thank you!
I’m glad to hear that.
Sorry, I don’t use notebooks, I find that they confuse beginners and hide / cause errors.
I write code and words in a text editor and post to my blog.
is there a way to download tensor flow a 32 bit windows computer?
Perhaps this tutorial will help:
https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/
what is the actual difference between neural network learning and deep neural network(deep learning) learning.?? is it only in weight initialization in each layer? or in back propagation algorithm??
thanks in advance
No difference anymore:
https://machinelearningmastery.com/what-is-deep-learning/
sir what is the best library. theano or tensorflow ??. and what is the recommend library for the deep learning beginners??
Keras is the best place to start:
https://machinelearningmastery.com/start-here/#deeplearning
hello Jason,
I am working on a small project where I need to detect the emotion using 3d camera (from microsoft). I will be using CNN for this..but is it possible to use keras in this case.
It would be in real time.
Sure.
Can I install keras on 32 bit linux ?
Perhaps try this tutorial:
https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/
Hello Jason,
How can i load set of jpg images to be my dataset in the tensorflow codes.
Can you help me to find a documented code for that?
Sorry, I don’t have material on tensorflow nor on loading jpegs.
How would I fit a model to convergence (i.e. until the loss doesn’t change) instead of fitting it based on the epoch parameter?
Set a large number of epochs and use early stopping to stop training when there are no further improvements on a hold out dataset.
Hi Jason, thanks for your great articles. is that possible to use CNN networks without GPU and external hardware?
Yes. all of the CNN examples on this blog will run on CPU or GPU.
Some large examples might just take a long time to run.
sir ,keras preparing transaction done but verifying transaction is always failed.what can i do ?
What do you mean by “verifying translation”?
Hello Jason
One maybe dumb, but simple question
Lets say that I run the CNN for 1000 epochs and I discover that maximum accuracy is obtained at epoch 534
Should I then re-run the model using 534 epochs so that I can then use it for prediction??
Also, a related question..
Is the accuracy of the model, the accuracy of its last epoch run?
You can, or you can use a checkpoint callback to save the best model during training automatically:
https://machinelearningmastery.com/check-point-deep-learning-models-keras/
Dear Dr Jason,
I just updated my version of Keras as at 12/10/2019 at 1218,
Imported keras
So the latest version is 2.3.1
Thank you
Anthony of Sydney
Nice work!