Last Updated on December 20, 2019

TensorFlow is a Python library for fast numerical computing created and released by Google.

It is a foundation library that can be used to create Deep Learning models directly or by using wrapper libraries that simplify the process built on top of TensorFlow.

In this post you will discover the TensorFlow library for Deep Learning.

**Kick-start your project** with my new book Deep Learning With Python, including *step-by-step tutorials* and the *Python source code* files for all examples.

Let’s get started.

## What is TensorFlow?

TensorFlow is an open source library for fast numerical computing.

It was created and is maintained by Google and released under the Apache 2.0 open source license. The API is nominally for the Python programming language, although there is access to the underlying C++ API.

Unlike other numerical libraries intended for use in Deep Learning like Theano, TensorFlow was designed for use both in research and development and in production systems, not least RankBrain in Google search and the fun DeepDream project.

It can run on single CPU systems, GPUs as well as mobile devices and large scale distributed systems of hundreds of machines.

## How to Install TensorFlow

Installation of TensorFlow is straightforward if you already have a Python SciPy environment.

TensorFlow works with Python 2.7 and Python 3.3+. You can follow the Download and Setup instructions on the TensorFlow website. Installation is probably simplest via PyPI and specific instructions of the pip command to use for your Linux or Mac OS X platform are on the Download and Setup webpage.

There are also virtualenv and docker images that you can use if you prefer.

To make use of the GPU, only Linux is supported and it requires the Cuda Toolkit.

## Your First Examples in TensorFlow

Computation is described in terms of data flow and operations in the structure of a directed graph.

**Nodes**: Nodes perform computation and have zero or more inputs and outputs. Data that moves between nodes are known as tensors, which are multi-dimensional arrays of real values.**Edges**: The graph defines the flow of data, branching, looping and updates to state. Special edges can be used to synchronize behavior within the graph, for example waiting for computation on a number of inputs to complete.**Operation**: An operation is a named abstract computation which can take input attributes and produce output attributes. For example, you could define an add or multiply operation.

### Computation with TensorFlow

This first example is a modified version of the example on the TensorFlow website. It shows how you can create a session, define constants and perform computation with those constants using the session.

1 2 3 4 5 |
import tensorflow as tf sess = tf.Session() a = tf.constant(10) b = tf.constant(32) print(sess.run(a+b)) |

Running this example displays:

1 |
42 |

### Linear Regression with TensorFlow

This next example comes from the introduction on the TensorFlow tutorial.

This examples shows how you can define variables (e.g. W and b) as well as variables that are the result of computation (y).

We get some sense of TensorFlow separates the definition and declaration of the computation from the execution in the session and the calls to run.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
import tensorflow as tf import numpy as np # Create 100 phony x, y data points in NumPy, y = x * 0.1 + 0.3 x_data = np.random.rand(100).astype(np.float32) y_data = x_data * 0.1 + 0.3 # Try to find values for W and b that compute y_data = W * x_data + b # (We know that W should be 0.1 and b 0.3, but Tensorflow will # figure that out for us.) W = tf.Variable(tf.random_uniform([1], -1.0, 1.0)) b = tf.Variable(tf.zeros([1])) y = W * x_data + b # Minimize the mean squared errors. loss = tf.reduce_mean(tf.square(y - y_data)) optimizer = tf.train.GradientDescentOptimizer(0.5) train = optimizer.minimize(loss) # Before starting, initialize the variables. We will 'run' this first. init = tf.initialize_all_variables() # Launch the graph. sess = tf.Session() sess.run(init) # Fit the line. for step in xrange(201): sess.run(train) if step % 20 == 0: print(step, sess.run(W), sess.run(b)) # Learns best fit is W: [0.1], b: [0.3] |

Running this example prints the following output:

1 2 3 4 5 6 7 8 9 10 11 |
(0, array([ 0.2629351], dtype=float32), array([ 0.28697217], dtype=float32)) (20, array([ 0.13929555], dtype=float32), array([ 0.27992988], dtype=float32)) (40, array([ 0.11148042], dtype=float32), array([ 0.2941364], dtype=float32)) (60, array([ 0.10335406], dtype=float32), array([ 0.29828694], dtype=float32)) (80, array([ 0.1009799], dtype=float32), array([ 0.29949954], dtype=float32)) (100, array([ 0.10028629], dtype=float32), array([ 0.2998538], dtype=float32)) (120, array([ 0.10008363], dtype=float32), array([ 0.29995731], dtype=float32)) (140, array([ 0.10002445], dtype=float32), array([ 0.29998752], dtype=float32)) (160, array([ 0.10000713], dtype=float32), array([ 0.29999638], dtype=float32)) (180, array([ 0.10000207], dtype=float32), array([ 0.29999897], dtype=float32)) (200, array([ 0.1000006], dtype=float32), array([ 0.29999971], dtype=float32)) |

You can learn more about the mechanics of TensorFlow in the Basic Usage guide.

## More Deep Learning Models

Your TensorFlow installation comes with a number of Deep Learning models that you can use and experiment with directly.

Firstly, you need to find out where TensorFlow was installed on your system. For example, you can use the following Python script:

1 |
python -c 'import os; import inspect; import tensorflow; print(os.path.dirname(inspect.getfile(tensorflow)))' |

For example, this could be:

1 |
/usr/lib/python2.7/site-packages/tensorflow |

Change to this directory and take note of the models subdirectory. Included are a number of deep learning models with tutorial-like comments, such as:

- Multi-threaded word2vec mini-batched skip-gram model.
- Multi-threaded word2vec unbatched skip-gram model.
- CNN for the CIFAR-10 network.
- Simple, end-to-end, LeNet-5-like convolutional MNIST model example.
- Sequence-to-sequence model with an attention mechanism.

Also check the examples directory as it contains an example using the MNIST dataset.

There is also an excellent list of tutorials on the main TensorFlow website. They show how to use different network types, different datasets and how to use the framework in various different ways.

Finally, there is the TensorFlow playground where you can experiment with small networks right in your web browser.

### Need help with Deep Learning in Python?

Take my free 2-week email course and discover MLPs, CNNs and LSTMs (with code).

Click to sign-up now and also get a free PDF Ebook version of the course.

## TensorFlow Resources

### More Resources

- TensorFlow Course on Udacity
- TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2015)

## Summary

In this post you discovered the TensorFlow Python library for deep learning.

You learned that it is a library for fast numerical computation, specifically designed for the types of operations that are required in the development and evaluation of large deep learning models.

Do you have any questions about TensorFlow or about this post? Ask your questions in the comments and I will do my best to answer them.

This was great help.

Can you post some more tutorials using tensor-flow.

Thanks.

I intend to soon, thanks Jatin.

print sess.run(a+b) this line is syntactically incorrect. print is a method so should have an opening and closing bracket.

Thanks Amit, fixed.

What if it’s Python 2.7? Isn’t the print statement without bracket valid?

The brackets are ignored/do nothing, and it makes the same code work in Python3.

Hi Jason

I could not find a models folder in my Tensorflow installation.

Can you please help?

I think they have been removed from the most recent release.

Hi Jason

In this page it’s written that “To make use of the GPU, only Linux is supported and it requires the Cuda Toolkit.” However, I think that Windows is also supported if Cuda toolkit and cudNN are installed. Is not it?

Thanks.

It may be, it did not appear to be the case at the time of writing.

When getting started in machine learning would you recommend ignoring Tensorflow for now sticking to your Getting Started regime?

Yes, in fact, for deep learning I would recommend using Keras that wraps the TensorFlow library.

Learn more here:

https://machinelearningmastery.com/faq/single-faq/what-deep-learning-library-do-you-recommend

Hi Jason, I have one question with respect to Tensor flow. I am trying to implement neural style transfer, using tensor flow. Was wondering if you may have any suggestions how to correct my error below. Thank you so much.

tf.reset_default_graph()

sess = tf.InteractiveSession()

sess.run(tf.initialize_all_variables())

sess.run(model[‘input’].assign(content_image))

Traceback (most recent call last):

File “”, line 1, in

sess.run(model[‘input’].assign(content_image))

File “/Users/sherrymukim/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py”, line 889, in run

run_metadata_ptr)

File “/Users/sherrymukim/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py”, line 1105, in _run

self._graph, fetches, feed_dict_tensor, feed_handles=feed_handles)

File “/Users/sherrymukim/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py”, line 414, in __init__

self._fetch_mapper = _FetchMapper.for_fetch(fetches)

File “/Users/sherrymukim/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py”, line 242, in for_fetch

return _ElementFetchMapper(fetches, contraction_fn)

File “/Users/sherrymukim/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py”, line 278, in __init__

‘Tensor. (%s)’ % (fetch, str(e)))

ValueError: Fetch argument cannot be interpreted as a Tensor. (Tensor Tensor(“Assign_5:0”, shape=(1, 300, 400, 3), dtype=float32_ref) is not an element of this graph.)

Sorry, I don’t have examples of tensorflow or style transfer. I cannot give you good advice.

Perhaps post to stackoverflow?

Hello,

So, do I have to mention that the program shoulb be run on tensorflow CPU support or is it implicit when I do not have a GPU?

Also, I am new to deep learning and all this API, CUDA, KERAS… etc are confusing. what are all this?

Regards

No Keras will use TensorFlow however it is configured. TensorFlow must be configured to use the GPU and I do not cover how to do that sorry.

Hi Jason!

As always, thank you for taking the time and energy in this wonderful Website!!!

From this section of your Linear regression example:

I get the following error:

NameError Traceback (most recent call last)

in

27

28 # Fit the line.

—> 29 for step in xrange(201):

30 sess.run(train)

31 if step % 20 == 0:

NameError: name ‘xrange’ is not defined

Perhaps confirm that you have Python 2.7 or 3.5+ installed?

Jose….remove “x”…just use range(201)…

I tried many times, but still the same issue, before that, I was using TensorFlow on the same PC. Just I have installed a window and tried to install again TensorFlow but getting an error.

ImportError Traceback (most recent call last)

~\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py in

57

—> 58 from tensorflow.python.pywrap_tensorflow_internal import *

59 from tensorflow.python.pywrap_tensorflow_internal import __version__

~\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py in

27 return _mod

—> 28 _pywrap_tensorflow_internal = swig_import_helper()

29 del swig_import_helper

~\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py in swig_import_helper()

23 try:

—> 24 _mod = imp.load_module(‘_pywrap_tensorflow_internal’, fp, pathname, description)

25 finally:

~\Miniconda3\lib\imp.py in load_module(name, file, filename, details)

241 else:

–> 242 return load_dynamic(name, filename, file)

243 elif type_ == PKG_DIRECTORY:

~\Miniconda3\lib\imp.py in load_dynamic(name, path, file)

341 name=name, loader=loader, origin=path)

–> 342 return _load(spec)

343

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last)

in

—-> 1 import tensorflow as tf

~\Miniconda3\lib\site-packages\tensorflow\__init__.py in

26

27 # pylint: disable=g-bad-import-order

—> 28 from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import

29 from tensorflow.python.tools import module_util as _module_util

30

~\Miniconda3\lib\site-packages\tensorflow\python\__init__.py in

47 import numpy as np

48

—> 49 from tensorflow.python import pywrap_tensorflow

50

51 # Protocol buffers

~\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py in

72 for some common reasons and solutions. Include the entire stack trace

73 above this error message when asking for help.””” % traceback.format_exc()

—> 74 raise ImportError(msg)

75

76 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

ImportError: Traceback (most recent call last):

File “C:\Users\Khalil\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow.py”, line 58, in

from tensorflow.python.pywrap_tensorflow_internal import *

File “C:\Users\Khalil\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”, line 28, in

_pywrap_tensorflow_internal = swig_import_helper()

File “C:\Users\Khalil\Miniconda3\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py”, line 24, in swig_import_helper

_mod = imp.load_module(‘_pywrap_tensorflow_internal’, fp, pathname, description)

File “C:\Users\Khalil\Miniconda3\lib\imp.py”, line 242, in load_module

return load_dynamic(name, filename, file)

File “C:\Users\Khalil\Miniconda3\lib\imp.py”, line 342, in load_dynamic

return _load(spec)

ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions. Include the entire stack trace

above this error message when asking for help.

Sorry, I am not familiar with this error.

Perhaps try re-installing with Anaconda:

https://machinelearningmastery.com/setup-python-environment-machine-learning-deep-learning-anaconda/

Hi Jason,

I tried to follow this tensorflow tutorial to remember the “complex structure” of tensorflow works (tf 1.x version vs. Keras) and here are my main conclusions to troubles founded:

1) Due to current tensorflow version it is 2.x, it is “eager executed”, so method s such as sessio() and run() are not directly available, your linear regression study case written for tensorflow 1.x, must be upgrade in the following ways:

1.1) Or implementing under tf 2.x ‘eager executed’, I concluded that all your old code lines must be putted under a nest , such as :

with tf.compat.v1.Session() as sess:

…

in addition of updating some tensors with method: “tf.compat.v1”, for tensors in v2, such as:

tf.compat.v1.Session()

f.compat.v1.train.GradientDescentOptimizer(0.5)

tf.compat.v1.global_variables_initializer()

1.2) Or implementing tf 2.x by disabling “eager execution”, via starting your old code by the initial sentence:

tf.compat.v1.disable_eager_execution()

in both cases I got your same results

1.3) Or I also tried to use directly the new methods of tf 2.x version of ‘eager training’ translating your old code such as :

with tf.GradientTape() as tape:

…

but apply to a more ‘complex structure’ such as new loss, and grad = tape.gradient(loss, w)… so I give up 🙂

1.4) I also see that concept coming from new tensorflow 2.x is trying to apply new simple ideas of keras (wrapper such as tf.keras…) and implementing ‘eager execution’, …but now it is confusing vs. tf 1.x version …So I do not know now who is going to apply directly tensorflow under this current state …:-)

regards,

JG

This tutorial is somewhat abandoned, I recommend this one:

https://machinelearningmastery.com/tensorflow-tutorial-deep-learning-with-tf-keras/

OK I will do it.

Anyway I was referring to apply pure Tensorflow to research and develop ML codes… but probably there is not much advantage, if anyone, in front of using friendly keras wrapper.

Upgrade it to TensorFlow 2.0

Thanks.

what is the best way to be an expert in tensorflow?

Practice a lot!