The amount of data you need depends both on the complexity of your problem and on the complexity of your chosen algorithm. This is a fact, but does not help you if you are at the pointy end of a machine learning project. A common question I get asked is: How much data do I […]

## A Gentle Introduction to Mini-Batch Gradient Descent and How to Configure Batch Size

Stochastic gradient descent is the dominant method used to train deep learning models. There are three main variants of gradient descent and it can be confusing which one to use. In this post, you will discover the one type of gradient descent you should use in general and how to configure it. After completing this […]

## 5 Examples of Simple Sequence Prediction Problems for Learning LSTM Recurrent Neural Networks

Sequence prediction is different from traditional classification and regression problems. It requires that you take the order of observations into account and that you use models like Long Short-Term Memory (LSTM) recurrent neural networks that have memory and that can learn any temporal dependence between observations. It is critical to apply LSTMs to learn how […]

## Gentle Introduction to Models for Sequence Prediction with Recurrent Neural Networks

Sequence prediction is a problem that involves using historical sequence information to predict the next value or values in the sequence. The sequence may be symbols like letters in a sentence or real values like those in a time series of prices. Sequence prediction may be easiest to understand in the context of time series […]

## What is the Difference Between Test and Validation Datasets?

A validation dataset is a sample of data held back from training your model that is used to give an estimate of model skill while tuning model’s hyperparameters. The validation dataset is different from the test dataset that is also held back from the training of the model, but is instead used to give an unbiased […]

## How to One Hot Encode Sequence Data in Python

Machine learning algorithms cannot work with categorical data directly. Categorical data must be converted to numbers. This applies when you are working with a sequence classification type problem and plan on using deep learning methods such as Long Short-Term Memory recurrent neural networks. In this tutorial, you will discover how to convert your input or […]

## How to Remove Trends and Seasonality with a Difference Transform in Python

Time series datasets may contain trends and seasonality, which may need to be removed prior to modeling. Trends can result in a varying mean over time, whereas seasonality can result in a changing variance over time, both which define a time series as being non-stationary. Stationary datasets are those that have a stable mean and […]

## How to Scale Data for Long Short-Term Memory Networks in Python

The data for your sequence prediction problem probably needs to be scaled when training a neural network, such as a Long Short-Term Memory recurrent neural network. When a network is fit on unscaled data that has a range of values (e.g. quantities in the 10s to 100s) it is possible for large inputs to slow […]

## A Tour of Recurrent Neural Network Algorithms for Deep Learning

Recurrent neural networks, or RNNs, are a type of artificial neural network that add additional weights to the network to create cycles in the network graph in an effort to maintain an internal state. The promise of adding state to neural networks is that they will be able to explicitly learn and exploit context in […]

## Gentle Introduction to the Adam Optimization Algorithm for Deep Learning

The choice of optimization algorithm for your deep learning model can mean the difference between good results in minutes, hours, and days. The Adam optimization algorithm is an extension to stochastic gradient descent that has recently seen broader adoption for deep learning applications in computer vision and natural language processing. In this post, you will […]