Hyperparameter optimization is a big part of deep learning. The reason is that neural networks are notoriously difficult to configure, and a lot of parameters need to be set. On top of that, individual models can be very slow to train. In this post, you will discover how to use the grid search capability from […]

# Author Archive | Jason Brownlee

## Evaluate the Performance of Deep Learning Models in Keras

Keras is an easy-to-use and powerful Python library for deep learning. There are a lot of decisions to make when designing and configuring your deep learning models. Most of these decisions must be resolved empirically through trial and error and by evaluating them on real data. As such, it is critically important to have a […]

## How to Build Multi-Layer Perceptron Neural Network Models with Keras

The Keras Python library for deep learning focuses on creating models as a sequence of layers. In this post, you will discover the simple components you can use to create neural networks and simple deep learning models using Keras from TensorFlow. Let’s get started. May 2016: First version Update Mar/2017: Updated example for Keras 2.0.2, […]

## How to Save and Load Your Keras Deep Learning Model

Keras is a simple and powerful Python library for deep learning. Since deep learning models can take hours, days, and even weeks to train, it is important to know how to save and load them from a disk. In this post, you will discover how to save your Keras models to files and load them […]

## Your First Deep Learning Project in Python with Keras Step-by-Step

Keras is a powerful and easy-to-use free open source Python library for developing and evaluating deep learning models. It is part of the TensorFlow library and allows you to define and train neural network models in just a few lines of code. In this tutorial, you will discover how to create your first deep learning neural […]

## Introduction to the Python Deep Learning Library TensorFlow

TensorFlow is a Python library for fast numerical computing created and released by Google. It is a foundation library that can be used to create Deep Learning models directly or by using wrapper libraries that simplify the process built on top of TensorFlow. In this post, you will discover the TensorFlow library for Deep Learning. […]

## Gradient Descent With AdaGrad From Scratch

Gradient descent is an optimization algorithm that follows the negative gradient of an objective function in order to locate the minimum of the function. A limitation of gradient descent is that it uses the same step size (learning rate) for each input variable. This can be a problem on objective functions that have different amounts […]

## Gradient Descent Optimization With AMSGrad From Scratch

Gradient descent is an optimization algorithm that follows the negative gradient of an objective function in order to locate the minimum of the function. A limitation of gradient descent is that a single step size (learning rate) is used for all input variables. Extensions to gradient descent like the Adaptive Movement Estimation (Adam) algorithm use […]

## Gradient Descent Optimization With AdaMax From Scratch

Gradient descent is an optimization algorithm that follows the negative gradient of an objective function in order to locate the minimum of the function. A limitation of gradient descent is that a single step size (learning rate) is used for all input variables. Extensions to gradient descent, like the Adaptive Movement Estimation (Adam) algorithm, use […]

## A Gentle Introduction to Premature Convergence

Convergence refers to the limit of a process and can be a useful analytical tool when evaluating the expected performance of an optimization algorithm. It can also be a useful empirical tool when exploring the learning dynamics of an optimization algorithm, and machine learning algorithms trained using an optimization algorithm, such as deep learning neural […]