Theano is a Python library for fast numerical computation that can be run on the CPU or GPU.
It is a key foundational library for Deep Learning in Python that you can use directly to create Deep Learning models or wrapper libraries that greatly simplify the process.
In this post you will discover the Theano Python library.
What is Theano?
Theano is an open source project released under the BSD license and was developed by the LISA (now MILA) group at the University of Montreal, Quebec, Canada (home of Yoshua Bengio). It is named after a Greek mathematician.
At it’s heart Theano is a compiler for mathematical expressions in Python. It knows how to take your structures and turn them into very efficient code that uses NumPy, efficient native libraries like BLAS and native code (C++) to run as fast as possible on CPUs or GPUs.
It uses a host of clever code optimizations to squeeze as much performance as possible from your hardware. If you are into the nitty-gritty of mathematical optimizations in code, check out this interesting list.
The actual syntax of Theano expressions is symbolic, which can be off putting to beginners used to normal software development. Specifically, expression are defined in the abstract sense, compiled and later actually used to make calculations.
It was specifically designed to handle the types of computation required for large neural network algorithms used in Deep Learning. It was one of the first libraries of its kind (development started in 2007) and is considered an industry standard for Deep Learning research and development.
How to Install Theano
Theano provides extensive installation instructions for the major operating systems: Windows, OS X and Linux. Read the Installing Theano guide for your platform.
Theano assumes a working Python 2 or Python 3 environment with SciPy. There are ways to make the installation easier, such as using Anaconda to quickly set up Python and SciPy on your machine as well as using Docker images.
With a working Python and SciPy environment, it is relatively straightforward to install Theano. from PyPI using pip, for example:
pip install Theano
At the time of writing the last official release of Theano was version 0.8 which was released 21th March 2016.
New releases may be announced and you will want to update to get any bug fixes and efficiency improvements. You can upgrade Theano using pip as follows:
sudo pip install --upgrade --no-deps theano
You may want to use the bleeding edge version of Theano checked directly out of Github.
This may be required for some wrapper libraries that make use of bleeding edge API changes. You can install Theano directly from a Github checkout as follows:
pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
You are now ready to run Theano on your CPU, which is just fine for the development of small models.
Large models may run slowly on the CPU. If you have a Nvidia GPU, you may want to look into configuring Theano to use your GPU. Read the Using the GPU guides for Linux or Mac OS X to set up Theano to use the GPU and the Using the GPU guide for how to test whether it is working.
Simple Theano Example
In this section we demonstrate a simple Python script that gives you a flavor of Theano.
It is taken from the Theano at a Glance guide. In this example we define two symbolic floating point variables a and b.
We define an expression that uses these variables (c = a + b).
We then compile this symbolic expression into a function using Theano that we can use later.
Finally, we use our complied expression by plugging in some real values and performing the calculation using efficient compiled Theano code under the covers.
from theano import tensor
# declare two symbolic floating-point scalars
a = tensor.dscalar()
b = tensor.dscalar()
# create a simple expression
c = a + b
# convert the expression into a callable object that takes (a,b)
# values as input and computes a value for c
f = theano.function([a,b], c)
# bind 1.5 to 'a', 2.5 to 'b', and evaluate 'c'
assert 4.0 == f(1.5, 2.5)
Running the example does not provide any output. The assertion that 1.5 + 2.5 = 4.0 is true.
This is a useful example as it gives you a flavor for how a symbolic expression can be defined, compiled and used. You can see how this may be scaled up to large vector and matrix operations required for deep learning.
Extensions and Wrappers for Theano
If you are new to deep learning you do not have to use Theano directly.
In fact, you are highly encouraged to use one of many popular Python projects that make Theano a lot easier to use for deep learning.
These projects provide data structures and behaviors in Python, specifically designed to quickly and reliably create deep learning models whilst ensuring that fast and efficient models are created and executed by Theano under the covers.
The amount of Theano syntax exposed by the libraries varies.
- For example the Lasagne library provides convenience classes for creating deep learning model but still expects you to know and make use of Theano syntax. This is good for beginners that know or are willing to learn a little Theano as well.
- Another example is Keras that hides Theano completely and provides a very simple API to work with to create Deep Learning models. It hides Theano so well, that it can in fact run as a wrapper for another popular foundation framework called TensorFlow.
I highly recommend experimenting a little with Theano directly, then selecting a wrapper library to learn and practice deep learning.
For a full list of libraries built on Theano see the Related projects guide on the Theano Wiki.
Beat the Math/Theory Doldrums and Start using Deep Learning in your own projects Today, without getting lost in “documentation hell”
Get my free Deep Learning With Python mini course and develop your own deep nets by the time you’ve finished the first PDF with just a few lines of Python.
Daily lessons in your inbox for 14 days, and a DL-With-Python “Cheat Sheet” you can download right now.
More Theano Resources
Looking for some more resources on Theano? Take a look at some of the following.
- Theano Official Homepage
- Theano GitHub Repository
- Theano: A CPU and GPU Math Compiler in Python (2010) (PDF)
- List of Libraries Built on Theano
- List of Theano configuration options
Theano and Deep Learning Tutorials
Getting Help with Theano
In this post you discovered the Theano Python library for efficient numerical computation.
You learned that it is a foundation library used for deep learning research and development and that it can be used directly to create deep learning models or by convenient libraries built on top of it such as Lasagne and Keras.
Do you have any questions about Theano or about deep learning in Python? Ask your question in the comments and I will do my best to answer it.
Frustrated With Your Progress In Deep Learning?
What If You Could Develop Your Own Deep Nets in Minutes
...with just a few lines of Python
Discover how in my new Ebook: Deep Learning With Python
It covers self-study tutorials and end-to-end projects on topics like:
Multilayer Perceptrons, Convolutional Nets and Recurrent Neural Nets, and more...
Finally Bring Deep Learning To
Your Own Projects
Skip the Academics. Just Results.