Taxonomy of Time Series Forecasting Problems

When you are presented with a new time series forecasting problem, there are many things to consider.

The choice that you make directly impacts each step of the project from the design of a test harness to evaluate forecast models to the fundamental difficulty of the forecast problem that you are working on.

It is possible to very quickly narrow down the options by working through a series of questions about your time series forecasting problem. By considering a few themes and questions within each theme, you narrow down the type of problem, test harness, and even choice of algorithms for your project.

In this post, you will discover a framework that you can use to quickly understand and frame your time series forecasting problem.

Kick-start your project with my new book Deep Learning for Time Series Forecasting, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

Taxonomy of Time Series Forecasting Problems

Taxonomy of Time Series Forecasting Problems
Photo by Adam Meek, some rights reserved.

Framework Overview

Time series forecasting involves developing and using a predictive model on data where there is an ordered relationship between observations.

You can learn more about what time series forecasting is in this post:

Before you get started on your project, you can answer a few questions and greatly improve your understanding of the structure of your forecast problem, the structure of the model requires, and how to evaluate it.

The framework presented in this post is divided into seven parts; they are:

  1. Inputs vs. Outputs
  2. Endogenous vs. Exogenous
  3. Unstructured vs. Structured
  4. Regression vs. Classification
  5. Univariate vs. Multivariate
  6. Single-step vs. Multi-step
  7. Static vs. Dynamic
  8. Contiguous vs. Discontiguous

I recommend working through this framework before starting any time series forecasting project.

Your answers may not be crisp on the first time through and the questions may require to you study the data, the domain, and talk to experts and stakeholders.

Update your answers as you learn more as it will help to keep you on track, avoid distractions, and develop the actual model that you need for your project.

Need help with Deep Learning for Time Series?

Take my free 7-day email crash course now (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

1. Inputs vs. Outputs

Generally, a prediction problem involves using past observations to predict or forecast one or more possible future observations.

The goal is to guess about what might happen in the future.

When you are required to make a forecast, it is critical to think about the data that you will have available to make the forecast and what you will be guessing about the future.

We can summarize this as what are the inputs and outputs of the model when making a single forecast.

  • Inputs: Historical data provided to the model in order to make a single forecast.
  • Outputs: Prediction or forecast for a future time step beyond the data provided as input.

The input data is not the data used to train the model. We are not at that point yet. It is the data used to make one forecast, for example the last seven days of sales data to forecast the next one day of sales data.

Defining the inputs and outputs of the model forces you to think about what exactly is or may be required to make a forecast.

You may not be able to be specific when it comes to input data. For example, you may not know whether one or multiple prior time steps are required to make a forecast. But you will be able to identify the variables that could be used to make a forecast.

What are the inputs and outputs for a forecast?

2. Endogenous vs. Exogenous

The input data can be further subdivided in order to better understand its relationship to the output variable.

An input variable is endogenous if it is affected by other variables in the system and the output variable depends on it.

In a time series, the observations for an input variable depend upon one another. For example, the observation at time t is dependent upon the observation at t-1; t-1 may depend on t-2, and so on.

An input variable is an exogenous variable if it is independent of other variables in the system and the output variable depends upon it.

Put simply, endogenous variables are influenced by other variables in the system (including themselves) whereas as exogenous variables are not and are considered as outside the system.

  • Endogenous: Input variables that are influenced by other variables in the system and on which the output variable depends.
  • Exogenous: Input variables that are not influenced by other variables in the system and on which the output variable depends.

Typically, a time series forecasting problem has endogenous variables (e.g. the output is a function of some number of prior time steps) and may or may not have exogenous variables.

Often, exogenous variables are ignored given the strong focus on the time series. Explicitly thinking about both variable types may help to identify easily overlooked exogenous data or even engineered features that may improve the model.

What are the endogenous and exogenous variables?

3. Regression vs. Classification

Regression predictive modeling problems are those where a quantity is predicted.

A quantity is a numerical value; for example a price, a count, a volume, and so on. A time series forecasting problem in which you want to predict one or more future numerical values is a regression type predictive modeling problem.

Classification predictive modeling problems are those where a category is predicted.

A category is a label from a small well-defined set of labels; for example {“hot”, “cold”}, {“up”, “down”}, and {“buy”, “sell”} are categories.

A time series forecasting problem in which you want to classify input time series data is a classification type predictive modeling problem.

  • Regression: Forecast a numerical quantity.
  • Classification: Classify as one of two or more labels.

Are you working on a regression or classification predictive modeling problem?

There is some flexibility between these types.

For example, a regression problem can be reframed as classification and a classification problem can be reframed as regression. Some problems, like predicting an ordinal value, can be framed as either classification and regression. It is possible that a reframing of your time series forecasting problem may simplify it.

What are some alternate ways to frame your time series forecasting problem?

4. Unstructured vs. Structured

It is useful to plot each variable in a time series and inspect the plot looking for possible patterns.

A time series for a single variable may not have any obvious pattern.

We can think of a series with no pattern as unstructured, as in there is no discernible time-dependent structure.

Alternately, a time series may have obvious patterns, such as a trend or seasonal cycles as structured.

We can often simplify the modeling process by identifying and removing the obvious structures from the data, such as an increasing trend or repeating cycle. Some classical methods even allow you to specify parameters to handle these systematic structures directly.

  • Unstructured: No obvious systematic time-dependent pattern in a time series variable.
  • Structured: Systematic time-dependent patterns in a time series variable (e.g. trend and/or seasonality).

Are the time series variables unstructured or structured?

5. Univariate vs. Multivariate

A single variable measured over time is referred to as a univariate time series. Univariate means one variate or one variable.

Multiple variables measured over time is referred to as a multivariate time series: multiple variates or multiple variables.

  • Univariate: One variable measured over time.
  • Multivariate: Multiple variables measured over time.

Are you working on a univariate or multivariate time series problem?

Considering this question with regard to inputs and outputs may add a further distinction. The number of variables may differ between the inputs and outputs, e.g. the data may not be symmetrical.

For example, you may have multiple variables as input to the model and only be interested in predicting one of the variables as output. In this case, there is an assumption in the model that the multiple input variables aid and are required in predicting the single output variable.

  • Univariate and Multivariate Inputs: One or multiple input variables measured over time.
  • Univariate and Multivariate Outputs: One or multiple output variables to be predicted.

6. Single-step vs. Multi-step

A forecast problem that requires a prediction of the next time step is called a one-step forecast model.

Whereas a forecast problem that requires a prediction of more than one time step is called a multi-step forecast model.

The more time steps to be projected into the future, the more challenging the problem given the compounding nature of the uncertainty on each forecasted time step.

  • One-Step: Forecast the next time step.
  • Multi-Step: Forecast more than one future time steps.

Do you require a single-step or a multi-step forecast?

7. Static vs. Dynamic

It is possible to develop a model once and use it repeatedly to make predictions.

Given that the model is not updated or changed between forecasts, we can think of this model as being static.

Conversely, we may receive new observations prior to making a subsequent forecast that could be used to create a new model or update the existing model. We can think of developing a new or updated model prior to each forecasts as a dynamic problem.

For example, it if the problem requires a forecast at the beginning of the week for the week ahead, we may receive the true observation at the end of the week that we can use to update the model prior to making next weeks forecast. This would be a dynamic model. If we do not get a true observation at the end of the week or we do and choose to not re-fit the model, this would be a static model.

We may prefer a dynamic model, but the constraints of the domain or limitations of a chosen algorithm may impose constraints that make this intractable.

  • Static. A forecast model is fit once and used to make predictions.
  • Dynamic. A forecast model is fit on newly available data prior to each prediction.

Do you require a static or a dynamically updated model?

8. Contiguous vs. Discontiguous

A time series where the observations are uniform over time may be described as contiguous.

Many time series problems have contiguous observations, such as one observation each hour, day, month or year.

A time series where the observations are not uniform over time may be described as discontiguous.

The lack of uniformity of the observations may be caused by missing or corrupt values. It may also be a feature of the problem where observations are only made available sporadically or at increasingly or decreasingly spaced time intervals.

In the case of non-uniform observations, specific data formatting may be required when fitting some models to make the observations uniform over time.

  • Contiguous. Observations are made uniform over time.
  • Discontiguous. Observations are not uniform over time.

Are your observations contiguous or discontiguous?

Further Reading

This section lists some resources for further reading.

Summary

To review, the themes and questions you can ask about your problem are as follows:

  1. Inputs vs. Outputs
    1. What are the inputs and outputs for a forecast?
  2. Endogenous vs. Exogenous
    1. What are the endogenous and exogenous variables?
  3. Unstructured vs. Structured
    1. Are the time series variables unstructured or structured?
  4. Regression vs. Classification
    1. Are you working on a regression or classification predictive modeling problem?
    2. What are some alternate ways to frame your time series forecasting problem?
  5. Univariate vs. Multivariate
    1. Are you working on a univariate or multivariate time series problem?
  6. Single-step vs. Multi-step
    1. Do you require a single-step or a multi-step forecast?
  7. Static vs. Dynamic
    1. Do you require a static or a dynamically updated model?
  8. Contiguous vs. Discontiguous
    1. Are your observations contiguous or discontiguous?

Did you find this framework useful for your time series forecasting problem?
Let me know in the comments below.

Do you have any questions?
Ask your questions in the comments below and I will do my best to answer.

Develop Deep Learning models for Time Series Today!

Deep Learning for Time Series Forecasting

Develop Your Own Forecasting models in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Deep Learning for Time Series Forecasting

It provides self-study tutorials on topics like:
CNNs, LSTMs, Multivariate Forecasting, Multi-Step Forecasting and much more...

Finally Bring Deep Learning to your Time Series Forecasting Projects

Skip the Academics. Just Results.

See What's Inside

40 Responses to Taxonomy of Time Series Forecasting Problems

  1. Avatar
    James Adams August 8, 2018 at 9:51 am #

    Is there a flowchart or any other tool that helps make sense of the answers to the questions above? I.e. once we know the answers to the questions above how do we then choose an appropriate model for use with our data?

  2. Avatar
    Rui August 9, 2018 at 12:17 am #

    Nice post.

    In Multi-Step forecast I would add that we could predict de difference, integral, etc, of the variable to predict over the future time …

    There is one thing I did not understand 100% (I think) that is :

    Multivariate VS Multivariable terms ?

    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3518362/

    https://www.researchgate.net/post/Are_the_terms_multivariate_and_multivariable_the_same

    For instance the CO2 “multivariate” time serie you have as example, according to these links is “Multivariate” or “Multivariable” ?

    great work

  3. Avatar
    Manish August 10, 2018 at 1:49 pm #

    Hi Jason,

    What approach do you recommend to forecast Forex rates?

    I have historical data about USD-INR currency pair and I wish to predict short, medium and long term forecast of this rate.

  4. Avatar
    Mayuri Ketkar August 22, 2018 at 4:13 pm #

    Thank you for this amazing introduction of various keywords I have been reading about time series. I am starting off a small time series project of mine; following your blogs.

  5. Avatar
    Nathaniel November 4, 2018 at 12:33 pm #

    What forecast do you record for a multivariate, endogenous, contiguous time series in which subject matter experts could provide the percentage impact of future time series prediction based on product changes? Maybe a weighted moving avg of some sort?

    • Avatar
      Jason Brownlee November 5, 2018 at 6:08 am #

      I would recommend testing a suite of methods in order to discover what works best for the specific dataset.

  6. Avatar
    jessy April 23, 2019 at 8:23 pm #

    hi jason,
    forecasting time series data using LSTM .is that forecasting possible before fitting the model .model.fit()…I am using lstm model forecasting time series data.

    • Avatar
      Jason Brownlee April 24, 2019 at 7:57 am #

      No, a model must be fit before you can make a prediction.

  7. Avatar
    Inna June 3, 2019 at 5:06 pm #

    Hi, Jason!
    Thanks for your article! Where can I read more about discontiguous time series? I found more “discontinuous” then “discontiguous” in google((( May be you know some articles

    • Avatar
      Jason Brownlee June 4, 2019 at 7:47 am #

      I don’t have tutorials on the topic.

      You can try modeling the data as-is.
      You can try padding the data to make the intervals consistent.
      You can try using a masking layer to skip the padded values.
      You can try upsampling/downsampling the data.

      I hope that helps as a start.

  8. Avatar
    Hassan June 11, 2019 at 6:11 pm #

    Hi, Jason,
    Thank you for the article,

    Training a univariate time series window, aren’t all the variables with impact are already embedded in the tends and seasonality of the historical data used for training? Wouldn’t be run a univariate is sufficient enough?

    • Avatar
      Jason Brownlee June 12, 2019 at 7:52 am #

      What do you mean exactly? Perhaps you can elaborate?

  9. Avatar
    Hassan June 16, 2019 at 3:11 pm #

    Let’s assume in a univariate dataset of GDP from 2000 till 2018, and I want to predict the GDP from 2019 till 2025.

    1. How do I decide if I use univariate time series prediction or a multivariate?

    2. Considering the trend and seasonality of GDP historical data (2000-2018) is already occurred due to the effect of other variables impacted on the GDP through the time series such as (inflation, unemployment…etc), so conducting univariate ARIMA or SARIMA would be sufficient enough as it considers the AR,I,MA ?

    Thank you.

    • Avatar
      Jason Brownlee June 17, 2019 at 8:15 am #

      If you have a single variable, it is univariate, otherwise it is multivariate.

      If you have multiple variables, try modelling as univariate then multivariate and compare results.

      Test and see…

  10. Avatar
    Ameya August 27, 2019 at 1:08 pm #

    Hi Jason,

    Thanks for this blog, it really helped me understand my problem in the first place. However, I believe one addition can be made to this blog to make it better to understand – provide 2-3 examples at the very end, and walk over all the 8 questions for those examples. This way we can better appreciate how a certain kind of task falls in a certain category for each of the 8 questions

  11. Avatar
    srinu October 18, 2019 at 10:18 pm #

    Dear Jason…Thank you for your nice article.

    Explains the comparison between deep learning and machine learning in relation to their respective performance across increasing amounts of time series data?

    When the amount of data is increased, machine learning techniques are insufficient in terms of performance. Explain

    • Avatar
      Jason Brownlee October 19, 2019 at 6:36 am #

      The mapping of problems to techniques is not so clear.

      Sometimes simple models do well on lots of data, sometimes not. You must always test many methods and discover what is best for a specific dataset.

  12. Avatar
    RK December 17, 2019 at 4:02 pm #

    Hi Jason. Thanks for the great article. What would be steps to model unstructured data and how can we confirm if the time series in itself does not have any predictive power.

    • Avatar
      Jason Brownlee December 18, 2019 at 6:00 am #

      What do you mean by unstructured?

      If the data is not predictive, you need new/different data.

  13. Avatar
    Rajesh Swarnkar January 21, 2020 at 2:44 pm #

    Jason,
    Your blog is one of the reference for me. Thanks.

    I was wondering how to fill in EXOGENOUS VARIABLE with Web Traffic timeseries data.

    I have a web traffic data which is sampled at each hour in 1 day multiple by 7 days data for 5 weeks.

    I realize there is significantly less traffic on weekends.

    I do i model a series representing this exogenous variable in a 24*7*5 data set??

    Please help.

    • Avatar
      Jason Brownlee January 21, 2020 at 2:54 pm #

      Thanks.

      Yes, for most linear models this is the expectation.

      • Avatar
        Rajesh Swarnkar January 21, 2020 at 8:43 pm #

        I’m sorry for typos in there.

        What i meant is do i have to hot encode the exogenous variable as vector of 0 and 1s for say holidays and weekends for 24 data points each ?

        I am very beginner to time series forcasting using RNN, CNN.

  14. Avatar
    Senthilkumar Radhakrishnan April 6, 2020 at 10:02 pm #

    First of all ,you are doing great work Dr.Jason,

    Can you suggest how to identify some problem to be a time series approach ,for example: If i am to forecast the sales of a supermarket for the period of next three months.Hence if we are to forecast we use time series approach

    Likewise what are the credentials to know whether this should be solved through time series.

    Thanks in advance

  15. Avatar
    Brett Forbes October 7, 2020 at 11:13 am #

    Curve fitting is a useful technique for sure, but answers without reasons are only useful in a vanishingly small set of cases. At some stage people are going to get interested in causality instead of correlation. Sure. its an easy answer to curve fit, but it tells you bugger all about the underlying system dynamics, or the causal factors.

    I like your article, it shows deep knowledge in the curve fitting arena. Have you ever looked at mining coincidences, and then addressing causal sequence dependence? For example (https://www.mybytes.de/papers/moerchen06tdm.pdf)

  16. Avatar
    Brett Forbes October 7, 2020 at 2:04 pm #

    Note, while the coincidence mining is great for setting up independent sequences, the itemset mining approach of ending up with partial order sequences is pretty unsatisfactory. Instead consider the Heuristics Miner algorithm from the process mining field, which is the best way of handling causal sequences dependence with noisy data.

    In short one mines coincidences to determine the overlaps, then mines the dependency graph that describes sequences causality. Check out (https://research.tue.nl/nl/publications/process-mining-with-the-heuristicsminer-algorithm )

Leave a Reply