A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem

Real-world time series forecasting is challenging for a whole host of reasons not limited to problem features such as having multiple input variables, the requirement to predict multiple time steps, and the need to perform the same type of prediction for multiple physical sites.

In this post, you will discover a standardized yet complex time series forecasting problem that has these properties, but is small and sufficiently well understood that it can be used to explore and better understand methods for developing forecasting models on challenging datasets.

After reading this post, you will know:

  • The competition and motivation for addressing the air-quality dataset.
  • An overview of the defined prediction problem and the data challenges it covers.
  • A description of the free data files that you can download and start working with immediately.

Let’s get started.

A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem

A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem
Photo by someone, some rights reserved.

EMC Data Science Global Hackathon

The dataset was used as the center of a Kaggle competition.

Specifically, a 24-hour hackathon hosted by Data Science London and Data Science Global as part of a Big Data Week event, two organizations that don’t seem to exist now, 6 years later.

The competition involved a multi-thousand-dollar cash prize, and the dataset was provided by the Cook County, Illinois local government, suggesting all locations mentioned in the dataset are in that locality.

The motivation for the challenge is to develop a better model for predicting air quality, taken from the competition description:

The EPA’s Air Quality Index is used daily by people suffering from asthma and other respiratory diseases to avoid dangerous levels of outdoor air pollutants, which can trigger attacks. According to the World Health Organisation there are now estimated to be 235 million people suffering from asthma. Globally, it is now the most common chronic disease among children, with incidence in the US doubling since 1980.

The competition description suggests that winning models could be used as the basis for a new air-quality prediction system, although it is not clear if any models were ever transitioned for this purpose.

The competition was won by a Kaggle employee, Ben Hamner, who presumably did not collect the prize given the conflict of interest. Ben described his winning approach in the blog post titled “Chucking everything into a Random Forest: Ben Hamner on Winning The Air Quality Prediction Hackathon” and provided his code on GitHub.

There is also a good discussion of solutions and related code in this forum post titled “General approaches to partitioning the models?“.

Predictive Modeling Problem

The data describes a multi-step forecasting problem given a multivariate time series across multiple sites or physical locations.

Given multiple weather measurements over time, predict a sequence of air quality measurements at specific future time intervals across multiple physical locations.

It is a challenging time series forecasting problem that has a lot of the qualities of real-world forecasting:

  • Incomplete data. Not all weather and air quality measures are available for all locations.
  • Missing data. Not all available measures have a complete history.
  • Multivariate inputs: The model inputs for each forecast are comprised of multiple weather observations.
  • Multi-step outputs: The model outputs are a discontiguous sequence of forecasted air quality measures.
  • Multi-site outputs: The mode must output a multi-step forecast for multiple physical sites.

Download the Dataset Files

The dataset is available for free from the Kaggle website.

You must create an account and sign-in with Kaggle before you can get access to download the dataset.

The dataset can be downloaded from here:

Description of the Dataset Files

There are 4 files of interest that you must download separately; they are:

File: SiteLocations.csv

This file contains a list of site locations marked by unique identifiers and their precise location on Earth measured by longitude and latitude.

All coordinates appear to be relatively close in the North-Western Hemisphere, e.g. America.

Below is a sample of the file.

File: SiteLocations_with_more_sites.csv

This file has the same format as SiteLocations.csv and appears to list all of the same locations as that file with some additional locations.

As the filename suggests, it is just an updated version of the list of sites.

Below is a sample of the file.

File: TrainingData.csv

This file contains the training data for modeling.

The data is presented in an unnormalized manner. Each row of data contains one set of meteorological measurements for one hour across multiple locations as well as the targets or outcomes for each location for that hour.

The measures include:

  • Time information, including the block of time, the index within the contiguous block of time, the average month, day of the week, and hour of the day.
  • Wind measurements such as direction and speed.
  • Temperature measurements such as minimum and maximum ambient temperature.
  • Pressure measurements such as minimum and maximum barometric pressure.

The target variables are a collection of different air quality or pollution measures at different physical locations.

Not all locations have all weather measurements and not all locations are concerned with all target measures. Further, for those recorded variables, there are missing values marked as NA.

Below is a sample of the file.

File: SubmissionZerosExceptNAs.csv

This file contains a sample of the submission for the prediction problem.

Each row specifies the prediction for each target measure across all target locations for a given hour in a chunk of contiguous time.

Below is a sample of the file.

Framing the Prediction Problem

A large part of the challenge of this prediction problem is the vast number of ways that the problem can be framed for modeling.

This is challenging because it is not clear which framing may be the best for this specific modeling problem.

For example, below are some questions to provoke thought about how the problem could be framed.

  • Is it better to impute or ignore missing observations?
  • Is it better to feed in a time series of weather observations or only the observations for the current hour?
  • Is it better to use weather observations from one or multiple source locations for a forecast?
  • Is it better to have one model for each location or one mode for all locations?
  • Is it better to have one model for each forecast time or one for all forecast times?

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Summary

In this post, you discovered the Kaggle air-quality dataset that provides a standard dataset for complex time series forecasting.

Specifically, you learned:

  • The competition and motivation for addressing the air-quality dataset.
  • An overview of the defined prediction problem and the data challenges it covers.
  • A description of the free data files that can download and start working with immediately.

Have you worked on this dataset, or do you intend to?
Share your experiences in the comments below.

Want to Develop Time Series Forecasts with Python?

Introduction to Time Series Forecasting With Python

Develop Your Own Forecasts in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Introduction to Time Series Forecasting With Python

It covers self-study tutorials and end-to-end projects on topics like:
Loading data, visualization, modeling, algorithm tuning, and much more...

Finally Bring Time Series Forecasting to
Your Own Projects

Skip the Academics. Just Results.

Click to learn more.

23 Responses to A Standard Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem

  1. Paris Tzou January 19, 2018 at 9:13 am #

    In this book, “Multivariate” is not dealt with.

    • Jason Brownlee January 20, 2018 at 8:12 am #

      How so?

      Multiple inputs or multiple input series is multivariate.

      • Tom April 5, 2018 at 1:35 am #

        I believe he is referring to your book ad at the end of the post.

        • Jason Brownlee April 5, 2018 at 6:13 am #

          Yes. I hope to write a book dedicated to the topic next.

  2. Hitendra Khairnar January 19, 2018 at 5:44 pm #

    How road traffic path recommendation problem can be framed as Multivariate, Multi-Step, and Multi-Site Time Series Forecasting Problem? I m thinking about different input parameters as follows 1. Distance between source and destination 2. Speed and type of Vehicle 3. Traffic volume and desnsity 4. Environmental conditions 5. Infrastructure conditions etc. and looking for suitable path for travelling from source to destination with minimum distance, feasible speed etc. Are there any data sets related to traffic data are available? Please suggest. Thanks

  3. Tobi Adeyemi January 23, 2018 at 8:23 pm #

    Hi Jason, it will be interesting to see how you will approach this problem. Do you think a reliable predictive model can be built from a dataset with this much missing variables? I look forward to your feedback.

    • Jason Brownlee January 24, 2018 at 9:54 am #

      Perhaps, it really depends on the specifics of the data.

  4. Alex Ludert January 28, 2018 at 5:04 am #

    Hi Jason!

    Great post! I’m trying to immitate this analysis in python and am having trouble dealing with the missing values in the random forest regression. From what I’m reading, the random forest can deal with the NaNs for the tree splitting but not for the regression.

    Looking at Ben’s matlab code it seems he’s “imputing” the missing values to 0. Am I correct in this? I’m not sure imputing the missing values to 0 for a regression is best. Any thoughts?

    Thank you in advance.

    • Jason Brownlee January 28, 2018 at 8:27 am #

      It really depends on the specific implementation.

      Perhaps try the xgboost library? or the sklearn implementation?

  5. karamba February 14, 2018 at 11:15 am #

    Hi Jason,

    Recently I have read Your article on this blog. Since that time I have read a lot of Your articles published here. Your knowledge and expierience is impressive.

    I’m trying to solve a multivariete (multiple input data) multi-step (multiple output) forecasting problem and I don’t know, which method should I use.

    I have time series data that represent weather(temperature, pressure, humidity, windspeed, wind direction etc.); pollution(pm10,pm2.5, NO2 etc.); traffic(average speed, street traffic congestion factor). I have a data from about ~100 localization inside middle-size city (327 km^2). Measurements are peformed each hour.

    I’m trying to implement a software which enable to perform 24h prediction of pollution for speciffic pollutant (for example pm10) using current data and data from history. Is there any preffered method to peform these type of analysis? Initially, I took into account two types of methods

    a) Multivariate Time Series Forecasting with LSTMs
    b) Method using RandomForest algorithms described in this article

    Maybe there are other methods I have not heard about. If You had to solve this type of problem, what methods would You use?

    Thanks for the work You do on this blog 🙂

    • Jason Brownlee February 14, 2018 at 2:43 pm #

      I would recommend a deep MLP on this type of problem.

      • karamba February 15, 2018 at 2:50 am #

        Thanks a lot.

        I have one more question. Is there any article on Your blog about making multivariete/multi step timeseries forecasting using MLP?

        I was trying to find some materials on the Internet about this method, but i canno’t find anything interesting. I have seen some scientifc papers, but none of them discribed exact the same problem.

        • Jason Brownlee February 15, 2018 at 8:47 am #

          I hope to write a book on this topic next.

          • karamba February 15, 2018 at 6:10 pm #

            It would be a very good idea. I would like to read such a book.

            Regardless, I will try to solve this problem myself. I just thought you had materials that might be useful to me, but now I understand why You have not published them yet.

            I am grateful for the information provided so far. Good luck 🙂

          • Soren P March 14, 2018 at 2:19 am #

            Hi Jason, what is your ETA for this book covering deep deep MLP for multivariate time series forecasting?

            I bet fixed windowed data i key, what your take on that?

            I know its VERT implementation dependent, but in these cases just how deep is deep in your view?

          • Jason Brownlee March 14, 2018 at 6:32 am #

            No fixed ETA at this stage. In progress.

  6. LuoXin April 15, 2018 at 11:43 am #

    Hi Jason:
    Thank you very much for your share in the LSTM timeseries prediction,it help me a lot in learning LSTM.But I still have some confusion about several time steps prediction by LSTM.
    The problem is that I have 12 years one dim timeseries data and want to use 84 hours data to predict the next 12 hours data by BP network and LSTM.I find the BP have the better performance,but I think the LSTM will do better,perhaps I have something wrong,could you tell me how to build a multi-steps prediction LSTM network?
    I am looking forward to hearing from your!

  7. song June 4, 2018 at 6:03 pm #

    Hi Jason,
    Thank you very much for your share in the LSTM timeseries prediction,which help me a lot in learning time series prediction. However, I still have some problems with time series prediction.They are listed as follows:
    1) I want to know the difference between sequence prediction and time series prediction . Whether can we use the model that deal with sequence prediction to make time series prediction or not?
    2)As I saw your past blogs, you have apply lstm model to Shampoo-Sales dataset which just contain 36 records. As we know, deep learning model are often used on big dataset. So, your experience may get poor performance. I have a dataset of 1000 records. In this case, I can use lstm to build model?
    3)Can you privide you some useful examples of variable time steps of lstm?

    I hope to receive your reply.

    • Jason Brownlee June 5, 2018 at 6:36 am #

      Good questions.

      Time series is a type of sequence prediction. Sequence prediction models can be used for time series.

      LSTMs may be good for your problem, but I would encourage you to test a suite of other methods as well to confirm that the LSTMs are skillful.

      When working with LSTMs with a variable number of input time steps, you must pad the inputs and use a masking layer to ignore the padding. I have examples on the blog.

      Does that help?

  8. Dwayne June 26, 2018 at 9:57 am #

    Hey Jason!
    Could you elabaorate more on using random forest to do a multi step time series forecast in Python? One of your further reading topics covers this on matlab. However, I don’t have a lot of experience in matlab and I am finding it difficult to understand.

Leave a Reply