Archive | Deep Learning for Time Series

A Gentle Introduction to Linear Algebra

How to Handle Missing Timesteps in Sequence Prediction Problems with Python

It is common to have missing observations from sequence data. Data may be corrupt or unavailable, but it is also possible that your data has variable length sequences by definition. Those sequences with fewer timesteps may be considered to have missing values. In this tutorial, you will discover how you can handle data with missing […]

Continue Reading 92
On the Suitability of Long Short-Term Memory Networks for Time Series Forecasting

On the Suitability of Long Short-Term Memory Networks for Time Series Forecasting

Long Short-Term Memory (LSTM) is a type of recurrent neural network that can learn the order dependence between items in a sequence. LSTMs have the promise of being able to learn the context required to make predictions in time series forecasting problems, rather than having this context pre-specified and fixed. Given the promise, there is […]

Continue Reading 84
The Promise of Recurrent Neural Networks for Time Series Forecasting

The Promise of Recurrent Neural Networks for Time Series Forecasting

Recurrent neural networks are a type of neural network that add the explicit handling of order in input observations. This capability suggests that the promise of recurrent neural networks is to learn the temporal context of input sequences in order to make better predictions. That is, that the suite of lagged observations required to make […]

Continue Reading 74
How to Use Weight Regularization with LSTM Networks for Time Series Forecasting

Weight Regularization with LSTM Networks for Time Series Forecasting

Long Short-Term Memory (LSTM) models are a recurrent neural network capable of learning sequences of observations. This may make them a network well suited to time series forecasting. An issue with LSTMs is that they can easily overfit training data, reducing their predictive skill. Weight regularization is a technique for imposing constraints (such as L1 […]

Continue Reading 39
How to Use Dropout with LSTM Networks for Time Series Forecasting

Dropout with LSTM Networks for Time Series Forecasting

Long Short-Term Memory (LSTM) models are a type of recurrent neural network capable of learning sequences of observations. This may make them a network well suited to time series forecasting. An issue with LSTMs is that they can easily overfit training data, reducing their predictive skill. Dropout is a regularization method where input and recurrent […]

Continue Reading 47
Instability of Online Learning for Stateful LSTM for Time Series Forecasting

Instability of Online Learning for Stateful LSTM for Time Series Forecasting

Some neural network configurations can result in an unstable model. This can make them hard to characterize and compare to other model configurations on the same problem using descriptive statistics. One good example of a seemingly unstable model is the use of online learning (a batch size of 1) for a stateful Long Short-Term Memory […]

Continue Reading 33
Stateful and Stateless LSTM for Time Series Forecasting with Python

Stateful and Stateless LSTM for Time Series Forecasting with Python

The Keras Python deep learning library supports both stateful and stateless Long Short-Term Memory (LSTM) networks. When using stateful LSTM networks, we have fine-grained control over when the internal state of the LSTM network is reset. Therefore, it is important to understand different ways of managing this internal state when fitting and making predictions with […]

Continue Reading 77