A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem

Real-world time series forecasting is challenging for a whole host of reasons not limited to problem features such as having multiple input variables, the requirement to predict multiple time steps, and the need to perform the same type of prediction for multiple physical sites.

In this post, you will discover a standardized yet complex time series forecasting problem that has these properties, but is small and sufficiently well understood that it can be used to explore and better understand methods for developing forecasting models on challenging datasets.

After reading this post, you will know:

  • The competition and motivation for addressing the air-quality dataset.
  • An overview of the defined prediction problem and the data challenges it covers.
  • A description of the free data files that you can download and start working with immediately.

Kick-start your project with my new book Time Series Forecasting With Python, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem

A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem
Photo by someone, some rights reserved.

EMC Data Science Global Hackathon

The dataset was used as the center of a Kaggle competition.

Specifically, a 24-hour hackathon hosted by Data Science London and Data Science Global as part of a Big Data Week event, two organizations that don’t seem to exist now, 6 years later.

The competition involved a multi-thousand-dollar cash prize, and the dataset was provided by the Cook County, Illinois local government, suggesting all locations mentioned in the dataset are in that locality.

The motivation for the challenge is to develop a better model for predicting air quality, taken from the competition description:

The EPA’s Air Quality Index is used daily by people suffering from asthma and other respiratory diseases to avoid dangerous levels of outdoor air pollutants, which can trigger attacks. According to the World Health Organisation there are now estimated to be 235 million people suffering from asthma. Globally, it is now the most common chronic disease among children, with incidence in the US doubling since 1980.

The competition description suggests that winning models could be used as the basis for a new air-quality prediction system, although it is not clear if any models were ever transitioned for this purpose.

The competition was won by a Kaggle employee, Ben Hamner, who presumably did not collect the prize given the conflict of interest. Ben described his winning approach in the blog post titled “Chucking everything into a Random Forest: Ben Hamner on Winning The Air Quality Prediction Hackathon” and provided his code on GitHub.

There is also a good discussion of solutions and related code in this forum post titled “General approaches to partitioning the models?“.

Predictive Modeling Problem

The data describes a multi-step forecasting problem given a multivariate time series across multiple sites or physical locations.

Given multiple weather measurements over time, predict a sequence of air quality measurements at specific future time intervals across multiple physical locations.

It is a challenging time series forecasting problem that has a lot of the qualities of real-world forecasting:

  • Incomplete data. Not all weather and air quality measures are available for all locations.
  • Missing data. Not all available measures have a complete history.
  • Multivariate inputs: The model inputs for each forecast are comprised of multiple weather observations.
  • Multi-step outputs: The model outputs are a discontiguous sequence of forecasted air quality measures.
  • Multi-site outputs: The mode must output a multi-step forecast for multiple physical sites.

Download the Dataset Files

The dataset is available for free from the Kaggle website.

You must create an account and sign-in with Kaggle before you can get access to download the dataset.

The dataset can be downloaded from here:

Description of the Dataset Files

There are 4 files of interest that you must download separately; they are:

File: SiteLocations.csv

This file contains a list of site locations marked by unique identifiers and their precise location on Earth measured by longitude and latitude.

All coordinates appear to be relatively close in the North-Western Hemisphere, e.g. America.

Below is a sample of the file.

File: SiteLocations_with_more_sites.csv

This file has the same format as SiteLocations.csv and appears to list all of the same locations as that file with some additional locations.

As the filename suggests, it is just an updated version of the list of sites.

Below is a sample of the file.

File: TrainingData.csv

This file contains the training data for modeling.

The data is presented in an unnormalized manner. Each row of data contains one set of meteorological measurements for one hour across multiple locations as well as the targets or outcomes for each location for that hour.

The measures include:

  • Time information, including the block of time, the index within the contiguous block of time, the average month, day of the week, and hour of the day.
  • Wind measurements such as direction and speed.
  • Temperature measurements such as minimum and maximum ambient temperature.
  • Pressure measurements such as minimum and maximum barometric pressure.

The target variables are a collection of different air quality or pollution measures at different physical locations.

Not all locations have all weather measurements and not all locations are concerned with all target measures. Further, for those recorded variables, there are missing values marked as NA.

Below is a sample of the file.

File: SubmissionZerosExceptNAs.csv

This file contains a sample of the submission for the prediction problem.

Each row specifies the prediction for each target measure across all target locations for a given hour in a chunk of contiguous time.

Below is a sample of the file.

Framing the Prediction Problem

A large part of the challenge of this prediction problem is the vast number of ways that the problem can be framed for modeling.

This is challenging because it is not clear which framing may be the best for this specific modeling problem.

For example, below are some questions to provoke thought about how the problem could be framed.

  • Is it better to impute or ignore missing observations?
  • Is it better to feed in a time series of weather observations or only the observations for the current hour?
  • Is it better to use weather observations from one or multiple source locations for a forecast?
  • Is it better to have one model for each location or one mode for all locations?
  • Is it better to have one model for each forecast time or one for all forecast times?

Further Reading

This section provides more resources on the topic if you are looking to go deeper.


In this post, you discovered the Kaggle air-quality dataset that provides a standard dataset for complex time series forecasting.

Specifically, you learned:

  • The competition and motivation for addressing the air-quality dataset.
  • An overview of the defined prediction problem and the data challenges it covers.
  • A description of the free data files that can download and start working with immediately.

Have you worked on this dataset, or do you intend to?
Share your experiences in the comments below.

Want to Develop Time Series Forecasts with Python?

Introduction to Time Series Forecasting With Python

Develop Your Own Forecasts in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Introduction to Time Series Forecasting With Python

It covers self-study tutorials and end-to-end projects on topics like: Loading data, visualization, modeling, algorithm tuning, and much more...

Finally Bring Time Series Forecasting to
Your Own Projects

Skip the Academics. Just Results.

See What's Inside

35 Responses to A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem

  1. Avatar
    Paris Tzou January 19, 2018 at 9:13 am #

    In this book, “Multivariate” is not dealt with.

    • Avatar
      Jason Brownlee January 20, 2018 at 8:12 am #

      How so?

      Multiple inputs or multiple input series is multivariate.

      • Avatar
        Tom April 5, 2018 at 1:35 am #

        I believe he is referring to your book ad at the end of the post.

        • Avatar
          Jason Brownlee April 5, 2018 at 6:13 am #

          Yes. I hope to write a book dedicated to the topic next.

  2. Avatar
    Hitendra Khairnar January 19, 2018 at 5:44 pm #

    How road traffic path recommendation problem can be framed as Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem? I m thinking about different input parameters as follows 1. Distance between source and destination 2. Speed and type of Vehicle 3. Traffic volume and desnsity 4. Environmental conditions 5. Infrastructure conditions etc. and looking for suitable path for travelling from source to destination with minimum distance, feasible speed etc. Are there any data sets related to traffic data are available? Please suggest. Thanks

    • Avatar
      Jason Brownlee January 20, 2018 at 8:17 am #

      I don’t know of any sorry.

    • Avatar
      Sidhu February 27, 2020 at 5:18 pm #

      Hi Jason , thanks for your articles on sequence prediction using LSTM. I have a problem where multiple users are measured on 6 different metrics over time.
      I am thinking it as a ” Multiple Time Series ” problem.

      Specifically, here’s how the problem is.

      User Id Energy sleep quality nutritional quality …
      1 4 3 4
      1 5 3 2
      2 1 5 3
      2 4 3 2

      So , I need to consider the above sequence for each user Id and predict the next sequence.
      I want to know how can I proceed with a problem like this.
      Please help me with this.

  3. Avatar
    Tobi Adeyemi January 23, 2018 at 8:23 pm #

    Hi Jason, it will be interesting to see how you will approach this problem. Do you think a reliable predictive model can be built from a dataset with this much missing variables? I look forward to your feedback.

    • Avatar
      Jason Brownlee January 24, 2018 at 9:54 am #

      Perhaps, it really depends on the specifics of the data.

  4. Avatar
    Alex Ludert January 28, 2018 at 5:04 am #

    Hi Jason!

    Great post! I’m trying to immitate this analysis in python and am having trouble dealing with the missing values in the random forest regression. From what I’m reading, the random forest can deal with the NaNs for the tree splitting but not for the regression.

    Looking at Ben’s matlab code it seems he’s “imputing” the missing values to 0. Am I correct in this? I’m not sure imputing the missing values to 0 for a regression is best. Any thoughts?

    Thank you in advance.

    • Avatar
      Jason Brownlee January 28, 2018 at 8:27 am #

      It really depends on the specific implementation.

      Perhaps try the xgboost library? or the sklearn implementation?

  5. Avatar
    karamba February 14, 2018 at 11:15 am #

    Hi Jason,

    Recently I have read Your article on this blog. Since that time I have read a lot of Your articles published here. Your knowledge and expierience is impressive.

    I’m trying to solve a multivariete (multiple input data) multi-step (multiple output) forecasting problem and I don’t know, which method should I use.

    I have time series data that represent weather(temperature, pressure, humidity, windspeed, wind direction etc.); pollution(pm10,pm2.5, NO2 etc.); traffic(average speed, street traffic congestion factor). I have a data from about ~100 localization inside middle-size city (327 km^2). Measurements are peformed each hour.

    I’m trying to implement a software which enable to perform 24h prediction of pollution for speciffic pollutant (for example pm10) using current data and data from history. Is there any preffered method to peform these type of analysis? Initially, I took into account two types of methods

    a) Multivariate Time Series Forecasting with LSTMs
    b) Method using RandomForest algorithms described in this article

    Maybe there are other methods I have not heard about. If You had to solve this type of problem, what methods would You use?

    Thanks for the work You do on this blog 🙂

    • Avatar
      Jason Brownlee February 14, 2018 at 2:43 pm #

      I would recommend a deep MLP on this type of problem.

      • Avatar
        karamba February 15, 2018 at 2:50 am #

        Thanks a lot.

        I have one more question. Is there any article on Your blog about making multivariete/multi step timeseries forecasting using MLP?

        I was trying to find some materials on the Internet about this method, but i canno’t find anything interesting. I have seen some scientifc papers, but none of them discribed exact the same problem.

        • Avatar
          Jason Brownlee February 15, 2018 at 8:47 am #

          I hope to write a book on this topic next.

          • Avatar
            karamba February 15, 2018 at 6:10 pm #

            It would be a very good idea. I would like to read such a book.

            Regardless, I will try to solve this problem myself. I just thought you had materials that might be useful to me, but now I understand why You have not published them yet.

            I am grateful for the information provided so far. Good luck 🙂

          • Avatar
            Soren P March 14, 2018 at 2:19 am #

            Hi Jason, what is your ETA for this book covering deep deep MLP for multivariate time series forecasting?

            I bet fixed windowed data i key, what your take on that?

            I know its VERT implementation dependent, but in these cases just how deep is deep in your view?

          • Avatar
            Jason Brownlee March 14, 2018 at 6:32 am #

            No fixed ETA at this stage. In progress.

  6. Avatar
    LuoXin April 15, 2018 at 11:43 am #

    Hi Jason:
    Thank you very much for your share in the LSTM timeseries prediction,it help me a lot in learning LSTM.But I still have some confusion about several time steps prediction by LSTM.
    The problem is that I have 12 years one dim timeseries data and want to use 84 hours data to predict the next 12 hours data by BP network and LSTM.I find the BP have the better performance,but I think the LSTM will do better,perhaps I have something wrong,could you tell me how to build a multi-steps prediction LSTM network?
    I am looking forward to hearing from your!

  7. Avatar
    song June 4, 2018 at 6:03 pm #

    Hi Jason,
    Thank you very much for your share in the LSTM timeseries prediction,which help me a lot in learning time series prediction. However, I still have some problems with time series prediction.They are listed as follows:
    1) I want to know the difference between sequence prediction and time series prediction . Whether can we use the model that deal with sequence prediction to make time series prediction or not?
    2)As I saw your past blogs, you have apply lstm model to Shampoo-Sales dataset which just contain 36 records. As we know, deep learning model are often used on big dataset. So, your experience may get poor performance. I have a dataset of 1000 records. In this case, I can use lstm to build model?
    3)Can you privide you some useful examples of variable time steps of lstm?

    I hope to receive your reply.

    • Avatar
      Jason Brownlee June 5, 2018 at 6:36 am #

      Good questions.

      Time series is a type of sequence prediction. Sequence prediction models can be used for time series.

      LSTMs may be good for your problem, but I would encourage you to test a suite of other methods as well to confirm that the LSTMs are skillful.

      When working with LSTMs with a variable number of input time steps, you must pad the inputs and use a masking layer to ignore the padding. I have examples on the blog.

      Does that help?

  8. Avatar
    Dwayne June 26, 2018 at 9:57 am #

    Hey Jason!
    Could you elabaorate more on using random forest to do a multi step time series forecast in Python? One of your further reading topics covers this on matlab. However, I don’t have a lot of experience in matlab and I am finding it difficult to understand.

  9. Avatar
    César Bouyssi February 15, 2019 at 2:59 am #

    Hi Jason,
    I’ve been looking at your blog all day and found plenty of very interesting articles.
    I’m currently working on the following problem:

    Forecasting the power production (in different fields – solar, wind, etc), with historical production data. I was thinking to improve the predictions by using the dates as an input too (month can be a good information, for solar exposition, and for the wind). I was also thinking of using weather dataset as an additionnal input.

    Do you thing a multivariate, multistep time serie could be a good approach (maybe with a LSTM network) ?

  10. Avatar
    Laura March 20, 2019 at 8:13 pm #

    Hi Jason,

    I have a more general question. In one of your tutorials you said that one should make the time series stationary. If I have a multivariate and test each variable for stationarity and get the result that some variables should be differenced and some are already stationary, what would you do? Probably it is not reasonable to difference only some of the variables..?

    • Avatar
      Jason Brownlee March 21, 2019 at 8:05 am #

      Start by modeling the data directly, then try making each/all stationary and see how it impacts the models ability to learn the problem.

  11. Avatar
    Liuzy December 23, 2019 at 1:15 pm #

    Hello, I am a Chinese researcher and have a strong interest in applying machine learning algorithms in the field of air quality, but I cannot download the data you use. Can you send me a copy?
    thank you

    • Avatar
      Jason Brownlee December 24, 2019 at 6:37 am #

      You must sig-up to kaggle, then you can download the dataset.

      Sorry, I cannot send it to you.

  12. Avatar
    Rahul Joarder July 30, 2021 at 5:16 pm #

    Hi Jason,

    In general multi site Time series, can we forecast for the multiple sites using 1 singular model(I want to use LSTM), or would we needed diff models for diff sites(How practical would this approach be if there are 1000+ site like customer data)

  13. Avatar
    Rahul Joarder August 3, 2021 at 8:15 pm #

    How can we train a single model(LSTM or SARIMAX) on a group of sites or all the sites . We would have diff values(for different sites) for the same feature at a particular time step? Do you have a article on this as this approach is not clear to me? Or could you just give give a brief about it, I would be able to take on from there!

    Thanks anyways for for the quick reply! Love your articles!

Leave a Reply