It is common to have missing observations from sequence data. Data may be corrupt or unavailable, but it is also possible that your data has variable length sequences by definition. Those sequences with fewer timesteps may be considered to have missing values. In this tutorial, you will discover how you can handle data with missing […]

# Archive | Deep Learning for Time Series

## On the Suitability of Long Short-Term Memory Networks for Time Series Forecasting

Long Short-Term Memory (LSTM) is a type of recurrent neural network that can learn the order dependence between items in a sequence. LSTMs have the promise of being able to learn the context required to make predictions in time series forecasting problems, rather than having this context pre-specified and fixed. Given the promise, there is […]

## The Promise of Recurrent Neural Networks for Time Series Forecasting

Recurrent neural networks are a type of neural network that add the explicit handling of order in input observations. This capability suggests that the promise of recurrent neural networks is to learn the temporal context of input sequences in order to make better predictions. That is, that the suite of lagged observations required to make […]

## Multistep Time Series Forecasting with LSTMs in Python

The Long Short-Term Memory network or LSTM is a recurrent neural network that can learn and forecast long sequences. A benefit of LSTMs in addition to learning long sequences is that they can learn to make a one-shot multi-step forecast which may be useful for time series forecasting. A difficulty with LSTMs is that they […]

## Weight Regularization with LSTM Networks for Time Series Forecasting

Long Short-Term Memory (LSTM) models are a recurrent neural network capable of learning sequences of observations. This may make them a network well suited to time series forecasting. An issue with LSTMs is that they can easily overfit training data, reducing their predictive skill. Weight regularization is a technique for imposing constraints (such as L1 […]

## Dropout with LSTM Networks for Time Series Forecasting

Long Short-Term Memory (LSTM) models are a type of recurrent neural network capable of learning sequences of observations. This may make them a network well suited to time series forecasting. An issue with LSTMs is that they can easily overfit training data, reducing their predictive skill. Dropout is a regularization method where input and recurrent […]

## How to Configure Multilayer Perceptron Network for Time Series Forecasting

It can be difficult when starting out on a new predictive modeling project with neural networks. There is so much to configure, and no clear idea where to start. It is important to be systematic. You can break bad assumptions and quickly hone in on configurations that work and areas for further investigation likely to […]

## Instability of Online Learning for Stateful LSTM for Time Series Forecasting

Some neural network configurations can result in an unstable model. This can make them hard to characterize and compare to other model configurations on the same problem using descriptive statistics. One good example of a seemingly unstable model is the use of online learning (a batch size of 1) for a stateful Long Short-Term Memory […]

## Stateful and Stateless LSTM for Time Series Forecasting with Python

The Keras Python deep learning library supports both stateful and stateless Long Short-Term Memory (LSTM) networks. When using stateful LSTM networks, we have fine-grained control over when the internal state of the LSTM network is reset. Therefore, it is important to understand different ways of managing this internal state when fitting and making predictions with […]

## How to Use Features in LSTM Networks for Time Series Forecasting

The Long Short-Term Memory (LSTM) network in Keras supports multiple input features. This raises the question as to whether lag observations for a univariate time series can be used as features for an LSTM and whether or not this improves forecast performance. In this tutorial, we will investigate the use of lag observations as features […]