3 Must-Own Books for Deep Learning Practitioners

Developing neural networks is often referred to as a dark art.

The reason for this is that being skilled at developing neural network models comes from experience. There are no reliable methods to analytically calculate how to design a “good” or “best” model for your specific dataset. You must draw on experience and experiment in order to discover what works on your problem.

A lot of this experience can come from actually developing neural networks on test problems.

Nevertheless, many people have come before and recorded their discoveries, best practices, and preferred techniques. You can learn a lot about how to design and configure neural networks from some of the best books on the topic.

In this post, you will discover the three books that I recommend reading and having next to you when developing neural networks for your datasets.

Discover how to train faster, reduce overfitting, and make better predictions with deep learning models in my new book, with 26 step-by-step tutorials and full source code.

Let’s get started.

Three Recommended Books on Neural Networks

There are three books that I think you must own physical copies of if you are a neural network practitioner.

They are:

  1. Neural Networks for Pattern Recognition, 1995.
  2. Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, 1999.
  3. Deep Learning, 2016.

These books are references, not tutorials.

You dip into them again and again before and during projects to ensure that you are getting everything you can out of your data and models.

These are the books that I read and reference all the time. If you have books that you recommend when developing neural network models, please let me know in the comments below.

Now, let’s take a closer look at each book in turn.

Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition by Christopher Bishop was released in 1995.

Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition

This great book was followed about a decade later by the still classic textbook Pattern Recognition and Machine Learning (fondly referred to as PRML). Christopher Bishop is both a professor at the University of Edinburgh and a director at Microsoft’s Cambridge research lab.

This book is a classic in the field of neural networks. It is a handbook that handily captures both the state of theory at the time, and techniques that remain just as relevant today nearly 25 years later.

Although reading the book cover to cover will provide you a robust foundation, I’d instead encourage you to use it as a reference for getting the most out of your neural network models.

I’d recommend dipping into the following chapters as needed:

  • Chapter 7: Parameter Optimization Algorithms
  • Chapter 8: Pre-Processing and Feature Extraction
  • Chapter 9: Learning and Generalization.

Chapter 9 is worth the sticker price for the book alone, giving a laundry list of descriptions for regularization methods and ensemble methods you should be testing.

I recommend this book because given the description of new methods almost daily, practitioners often forget the tried and true basics.

I don’t think this book is in print anymore, but you can find secondhand and international versions everywhere online.

Want Better Results with Deep Learning?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Download Your FREE Mini-Course

Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks

Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks by Russell Reed and Robert Marks was released in 1999.

Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks

Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks

I have a large soft spot for this book.

I purchased it soon after it was released and used it as a reference for many of my own implementations of neural network algorithms through the 2000s.

There are two things I like the most about this book:

  • Code
  • Plots

The book uses mathematics and descriptions to explain concepts, but importantly they also use snippets of pseudocode or ANSI C to show how things work. This is invaluable the first time you’re coding backpropagation of error or an activation functions.

The book also uses plots of the decision surface models. This is invaluable to understand what the models are doing/seeing during training under different learning algorithms and how things like regularization effect the model.

There is perhaps an over focus on pruning methods given the authors interest in the area; nevertheless, I’d recommend dipping into the following chapters when developing your own models:

  • Chapter 14: Factors Influencing Generalization
  • Chapter 15: Generalization Prediction and Assessment
  • Chapter 16: Heuristics for Improving Generalization
  • Chapter 17: Effects of Training with Noisy Inputs

Although I recommend buying this book and having it next to you (always), Robert Marks has a reprint of the book on his website in HTML format:

Deep Learning

Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville was released in 2016.

Deep Learning

Deep Learning

This is the missing bridge between the classic books of the 1990s and modern deep learning.

Importantly, neural networks are introduced with careful mention of the innovations and milestones that have made the field into what it is today. Specifically Chapter 6: Deep Feedforward Networks and Section 6.6 Historical Notes.

There are three chapters that are must-reads for neural network practitioners; they are:

  • Chapter 7: Regularization for Deep Learning
  • Chapter 8: Optimization for Training Deep Models
  • Chapter 11: Practical Methodology

Chapter 11 especially is important as it ties together specific methods and how and when to use them in practice. It is by far worth the purchase price of the book alone.

This is a must have. You need a physical copy of this book. Nevertheless, the entire text is available on the books website here:

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Books

Additional Books

Additional Links

Summary

In this post, you discovered the three reference books that I think that every neural network practitioner must own.

Do you use one or more of these books yourself?
What chapters do you reference heavily?

Are there other books that you reference a lot?
Let me know below.


Develop Better Deep Learning Models Today!

Better Deep Learning

Train Faster, Reduce Overftting, and Ensembles

…with just a few lines of python code

Discover how in my new Ebook:
Better Deep Learning

It provides self-study tutorials on topics like: weight decay, batch normalization, dropout, model stacking and much more…

Bring better deep learning to your projects!

Skip the Academics. Just Results.

Click to learn more.


12 Responses to 3 Must-Own Books for Deep Learning Practitioners

  1. David January 14, 2019 at 5:54 am #

    Hi Jason,
    Thanks for the recommendations. What books would you recommend for more practical deep learning (ie. A little less theory and more coding with real data)?
    Thanks for everything,
    David

  2. Dennis Cartin January 14, 2019 at 5:21 pm #

    I feel more motivated to continue reading the Deep Learning book. At chapter 5 right now but will jump to chapter 7, 8 and 11 as Jason said it is a must read chapter.

    Thanks Jason as always.

  3. Rahul Sangole January 15, 2019 at 10:48 am #

    As always Jason, thanks for your posts.

  4. Dominic Mackenzie January 28, 2019 at 9:17 am #

    Why do you say that we need physical copies of these books? What extra value do you see in the physical copies over an ebook version?

    • Jason Brownlee January 28, 2019 at 11:45 am #

      I like both ebooks and physical copies for different reasons, but I find physical copies of technical books invaluable. It is something about knowing where in the physical book a topic is covered and being able to turn to it – to know which page and where on the page. To hold it.

      Only for reference information though. I don’t see the need for tutorial based material.

  5. Hapimorning February 5, 2019 at 11:02 pm #

    Really valuable information… thanks for the posts…

  6. Nick June 21, 2019 at 4:16 pm #

    What about the book (if it happens for you to know) “Neural and adaptive systems”, by Jose C. Principe, Neil R. Euliano, and W. Curt Lefebvre? Do you consider it as another one worthy to be in the list?

    I would like to have your opinion.

Leave a Reply