How to Make Manual Predictions for ARIMA Models with Python

The autoregression integrated moving average model or ARIMA model can seem intimidating to beginners.

A good way to pull back the curtain in the method is to to use a trained model to make predictions manually. This demonstrates that ARIMA is a linear regression model at its core.

Making manual predictions with a fit ARIMA models may also be a requirement in your project, meaning that you can save the coefficients from the fit model and use them as configuration in your own code to make predictions without the need for heavy Python libraries in a production environment.

In this tutorial, you will discover how to make manual predictions with a trained ARIMA model in Python.

Specifically, you will learn:

  • How to make manual predictions with an autoregressive model.
  • How to make manual predictions with a moving average model.
  • How to make predictions with an autoregression integrated moving average model.

Let’s dive in.

  • Updated Apr/2019: Updated the link to dataset.
How to Make Manual Predictions for ARIMA Models with Python

How to Make Manual Predictions for ARIMA Models with Python
Photo by Bernard Spragg. NZ, some rights reserved.

Minimum Daily Temperatures Dataset

This dataset describes the minimum daily temperatures over 10 years (1981-1990) in the city Melbourne, Australia.

The units are in degrees Celsius and there are 3,650 observations. The source of the data is credited as the Australian Bureau of Meteorology.

Download the dataset and place it into your current working directory with the filename “daily-minimum-temperatures.csv“.

The example below demonstrates how to load the dataset as a Pandas Series and graph the loaded dataset.

Running the example creates a line plot of the time series.

Minimum Daily Temperatures Dataset Plot

Minimum Daily Temperatures Dataset Plot

Stop learning Time Series Forecasting the slow way!

Take my free 7-day email course and discover how to get started (with sample code).

Click to sign-up and also get a free PDF Ebook version of the course.

Start Your FREE Mini-Course Now!

ARIMA Test Setup

We will use a consistent test harness to fit ARIMA models and evaluate their predictions.

First, the loaded dataset is split into a train and test dataset. The majority of the dataset is used to fit the model and the last 7 observations (one week) are held back as the test dataset to evaluate the fit model.

A walk-forward validation, or rolling forecast, method is used as follows:

  1. Each time step in the test dataset is iterated.
  2. Within each iteration, a new ARIMA model is trained on all available historical data.
  3. The model is used to make a prediction for the next day.
  4. The prediction is stored and the “real” observation is retrieved from the test set and added to the history for use in the next iteration.
  5. The performance of the model is summarized at the end by calculating the root mean squared error (RMSE) of all predictions made compared to expected values in the test dataset.

Simple AR, MA, ARMA and ARMA models are developed. They are unoptimized and are used for demonstration purposes. You will surely be able to achieve better performance with a little tuning.

The ARIMA implementation from the statsmodels Python library is used and AR and MA coefficients are extracted from the ARIMAResults object returned from fitting the model.

The ARIMA model supports forecasts via the predict() and the forecast() functions.

Nevertheless, we will make manual predictions in this tutorial using the learned coefficients.

This is useful as it demonstrates that all that is required from a trained ARIMA model is the coefficients.

The coefficients in the statsmodels implementation of the ARIMA model do not use intercept terms. This means we can calculate the output values by taking the dot product of the learned coefficients and lag values (in the case of an AR model) and lag residuals (in the case of an MA model). For example:

The coefficients of a learned ARIMA model can be accessed from aARIMAResults object as follows:

  • AR Coefficients: model_fit.arparams
  • MA Coefficients: model_fit.maparams

We can use these retrieved coefficients to make predictions using the following manual predict() function.

For reference, you may find the following resources useful:

Let’s look at some simple but specific models and how to make manual predictions with this test setup.

Autoregression Model

The autoregression model, or AR, is a linear regression model on the lag observations.

An AR model with a lag of k can be specified in the ARIMA model as follows:

In this example, we will use a simple AR(1) for demonstration purposes.

Making a prediction requires that we retrieve the AR coefficients from the fit model and use them with the lag of observed values and call the custom predict() function defined above.

The complete example is listed below.

Note that the ARIMA implementation will automatically model a trend in the time series. This adds a constant to the regression equation that we do not need for demonstration purposes. We turn this convenience off by setting the ‘trend’ argument in the fit() function to the value ‘nc‘ for ‘no constant‘.

The fit() function also outputs a lot of verbose messages that we can turn off by setting the ‘disp‘ argument to ‘False‘.

Running the example prints the prediction and expected value each iteration for 7 days. The final RMSE is printed showing an average error of about 1.9 degrees Celsius for this simple model.

Experiment with AR models with different orders, such as 2 or more.

Moving Average Model

The moving average model, or MA, is a linear regression model of the lag residual errors.

An MA model with a lag of k can be specified in the ARIMA model as follows:

In this example, we will use a simple MA(1) for demonstration purposes.

Much like above, making a prediction requires that we retrieve the MA coefficients from the fit model and use them with the lag of residual error values and call the custom predict() function defined above.

The residual errors during training are stored in the ARIMA model under the ‘resid‘ parameter of the ARIMAResults object.

The complete example is listed below.

Running the example prints the predictions and expected values each iteration for 7 days and ends by summarizing the RMSE of all predictions.

The skill of the model is not great and you can use this as an opportunity to explore MA models with other orders and use them to make manual predictions.

You can see how it would be straightforward to keep track of the residual errors manually outside of the ARIMAResults object as new observations are made available. For example:

Next, let’s put the AR and MA models together and see how we can perform manual predictions.

Autoregression Moving Average Model

We have now seen how we can make manual predictions for a fit AR and MA model.

These approaches can be put directly together to make manual predictions for a fuller ARMA model.

In this example, we will fit an ARMA(1,1) model that can be configured in an ARIMA model as ARIMA(1,0,1) with no differencing.

The complete example is listed below.

You can see that the prediction (yhat) is the sum of the dot product of the AR coefficients and lag observations and the MA coefficients and lag residual errors.

Again, running the example prints the predictions and expected values each iteration and the summary RMSE for all predictions made.

We can now add differencing and show how to make predictions for a complete ARIMA model.

Autoregression Integrated Moving Average Model

The I in ARIMA stands for integrated and refers to the differencing performed on the time series observations before predictions are made in the linear regression model.

When making manual predictions, we must perform this differencing of the dataset prior to calling the predict() function. Below is a function that implements differencing of the entire dataset.

A simplification would be to keep track of the observation at the oldest required lag value and use that to calculate the differenced series prior to prediction as needed.

This difference function can be called once for each difference required of the ARIMA model.

In this example, we will use a difference level of 1, and combine it with the ARMA example in the previous section to give us an ARIMA(1,1,1) model.

The complete example is listed below.

You can see that the lag observations are differenced prior to their use in the call to the predict() function with the AR coefficients. The residual errors will also be calculated with regard to these differenced input values.

Running the example prints the prediction and expected value each iteration and summarizes the performance of all predictions made.

Summary

In this tutorial, you discovered how to make manual predictions for an ARIMA model with Python.

Specifically, you learned:

  • How to make manual predictions for an AR model.
  • How to make manual predictions for an MA model.
  • How to make manual predictions for an ARMA and ARIMA model.

Do you have any questions about making manual predictions?
Ask your questions in the comments below and I will do my best to answer.

Want to Develop Time Series Forecasts with Python?

Introduction to Time Series Forecasting With Python

Develop Your Own Forecasts in Minutes

...with just a few lines of python code

Discover how in my new Ebook:
Introduction to Time Series Forecasting With Python

It covers self-study tutorials and end-to-end projects on topics like:
Loading data, visualization, modeling, algorithm tuning, and much more...

Finally Bring Time Series Forecasting to
Your Own Projects

Skip the Academics. Just Results.

Click to learn more.

52 Responses to How to Make Manual Predictions for ARIMA Models with Python

  1. MIC February 8, 2017 at 4:50 pm #

    Hi Jason,

    Thanks for this tutorial.
    Now the error occure as follow, Please advise me about it.
    I think also that the code do not have an error.

    —> 23 model_fit = model.fit(trend=’nc’, disp=False)

    ValueError: could not convert string to float: ?0.2

    thanks

    • Jason Brownlee February 9, 2017 at 7:22 am #

      Open the downloaded data file and delete all instances of the “?” character.

      • MIC February 10, 2017 at 11:56 am #

        It worked fine.
        I did not think that it was caused by converting the file to CSV.

        Thank you, Jason.

  2. Luca May 5, 2017 at 2:42 am #

    Hi Jason

    Really appreciate for this article, I’ve got one question: in this example, why do we use an iterative way to determine the ARIMA parameters, can we fix the model parameters before the loop in test dataset and then doing the validation process?

    Thanks a lot for some more insights on it.

    • Jason Brownlee May 5, 2017 at 7:33 am #

      I’m not sure what you mean by iterative? Can you please elaborate?

      • Luca May 6, 2017 at 12:20 am #

        Thanks for the feedback, Jason, what I meant before is:

        for t in range(len(test)):
        model = ARIMA(history, order=(1,1,1))

        we see that in each loop, we train the model again and get a new set of parameters. Why not train the model just based on the training set and all parameters in the model is fixed, then loop for all test set and validate the error for each of them.

        Thanks again for your reply.

        • Jason Brownlee May 6, 2017 at 7:46 am #

          You can, but if we have new data (e.g. it is the next month and a new observation is available) then we should use it.

          That is what we are simulating here. It is called walk forward validation:
          http://machinelearningmastery.com/backtest-machine-learning-models-time-series-forecasting/

          • Luca May 8, 2017 at 7:16 pm #

            Thanks a lot Jason.
            Nice example from that link. I find that there’re plenty of quite useful and interesting information from your posts, I will go through others and post questions (if I have).
            Again, thanks for the work, great job. 🙂

          • Jason Brownlee May 9, 2017 at 7:40 am #

            Thanks Luca!

  3. Hans June 15, 2017 at 11:37 pm #

    Let’s say I have two data points in a week recorded over several years, for example data from Monday and Friday. Would this affect the model? Would this be relevant for the difference function?

  4. Hans June 15, 2017 at 11:38 pm #

    I have several ARIMA tutorials read on this site. Some use the difference function some not- like the parameter tunings. When do I need the difference function.

  5. Luca June 20, 2017 at 6:28 pm #

    Hi Jason

    Since the dataset seems to have strong seasonality, in this case, do we need to first decompose the data by removing the seasonality factor, and then, applying models such as ARIMA?

    Thanks

  6. David Ravnsborg July 6, 2017 at 2:28 pm #

    Hi Jason,

    Thanks for all the ARIMA tutorials! I’m preparing to analyze some gait data from a biomechanics lab I worked in last summer and they’re helping me to get to hang of the model.

  7. Srini July 15, 2017 at 4:53 am #

    Hi Jason,

    Imagine a scenario where we are using the fitted ARIMA model i.e. the coefficients on a new dataset (out of sample). Calculating the AR part is easy. Use the previous data of length ‘p’ (history). Whereas for the MA part, one needs the residuals for past ‘q’ values. Without fitting for the new data seen, how are the the residuals calculated for the new data. Do we use the residuals found in the training data? This is useful for for ARIMA models with q>0 (i.e. having MA coefficients).

    • Jason Brownlee July 15, 2017 at 9:46 am #

      Good question. The ARIMA model will make these available in “model_fit.resid” I believe.

  8. buffy July 18, 2017 at 4:19 pm #

    thanks for post, it is helpful.
    But I have question, why the history data need to append test data for each loop:
    model = ARIMA(history, order=(1,0,0))
    obs = test[t]
    history.append(obs)

    If I do’t know the test data value, for example just predict 6 month or 1 years days(365 days range) value, how to use the model to predict. Just like machine learning, used the train set to train model, and predict new data.

  9. Kanav Kariya November 15, 2017 at 9:54 am #

    Hey Jason,

    Thanks a lot for these ARIMA posts, I know close to nothing about time series and I’m learning this stuff to work on a project to find a cross-correlation matrix. I need to pre-whiten the data and I used your tutorial to fit a model to one of my series. Next I have to apply this model (filter) to another series and I was wondering if you could elaborate on how to do that using predict. I’ve been scouring the internet for some material, and I don’t know what exactly the contents and format of the ‘params’ field is supposed to be.

    Thanks a lot!

  10. Maksouda February 27, 2018 at 8:10 pm #

    Bonjour, est-il possible d’intĂ©grer d’autres variables dans le modèle ARIMA pour faire de la prĂ©diction ?
    Je n etrouve pas de réponses concernant cette question.

    • Jason Brownlee February 28, 2018 at 6:03 am #

      Yes, it is called exogenous variables.

      Sorry, I don’t have an example.

  11. William Ford March 12, 2018 at 11:54 am #

    Hi! How can I predct for example 20 reading forward?
    mod =ARIMA(X, order=(self.p, self.d, self.q)
    res = mod.fit()
    res.forecast(20)

  12. Shital Bhojani March 19, 2018 at 8:15 pm #

    Hello Jason,

    Here in example you said 3650 observations and have taken last 7 days for test set.. I am lil confused about my data set. I have weekly data of 7 weather parameters for 30 years. How can I select the training and test set?

  13. Mia April 24, 2018 at 11:53 pm #

    I tried both your method to manually predict the series (which I think correct) and statsmodels’ .predict(). Do you have an explanation on why there are not exactly same? statsmodels is not that clear about how they perform predictions. Thanks!

  14. Paola July 23, 2018 at 8:27 pm #

    I have a question, how do they acf and pacf were obtained here? I have a similar dataset with almost 1100 observations, and based on my acf and pacf, I have a significant lag from 1 to 27 for the acf. I want to know also if there is a limitation for the ARIMA model when we have those high values. Thanks

  15. Great January 13, 2019 at 1:53 am #

    hi,I think there is a big error in the ARIMA model about the difference.

    When we use AIRMA(p,d,q), if d not equal to 0, the model has done the difference step.
    So the predict result data also has been diffenenced, so we need to restore the data.

    But the restored result (no difference) is wrong. The predict error is very big.

    In your many ARIMA examples, such as ARIMA(5,1,0), the d = 1, but I didn’t find the restored difference.
    If the ARIMA(5,0,0), it doesn’t need to restore the difference.
    The difference I mean the parameter d in the ARIMA(p,d,q).
    I think maybe the python model ARIMA has little error.

    • Jason Brownlee January 13, 2019 at 5:42 am #

      The model will invert the difference (if needed) when a prediction is made.

  16. aoyong April 3, 2019 at 2:11 am #

    Hi, Jason, I have a question about the train and validation.
    In the four examples, you split the data into train and test part. But in your prediction, you also add the data in test to the history. Is it OK to do in this way? According to my understanding, it is not right to use the data in test set.

    Another question is about the prediction. According to the document (https://www.statsmodels.org/dev/generated/statsmodels.tsa.arima_model.ARMAResults.forecast.html#statsmodels.tsa.arima_model.ARMAResults.forecast), we can predict the future in many steps (not only 1). Why you do not use the steps directly?

    • Jason Brownlee April 3, 2019 at 6:47 am #

      Yes, this is called walk forward validation, more here:
      https://machinelearningmastery.com/backtest-machine-learning-models-time-series-forecasting/

      • aoyong April 3, 2019 at 8:25 pm #

        I saw the link. It is detailed. I also checked other web pages. It seems that “walk forward validation ” is uncommon in machine learning. If we want to compare the result with other methods. What we should do?

        If we use other complex methods and use “walk forward validation”, it will wast a lot of time to train the model.

        Thanks.

        • Jason Brownlee April 4, 2019 at 7:50 am #

          It is common in time series forecasting problems.

          Other methods should use the approach, and if they do not, such as using cross validation, their results are almost certainly invalid.

  17. Naveksha Sood April 26, 2019 at 8:04 pm #

    How can I calculate training error for time series?

    • Jason Brownlee April 27, 2019 at 6:29 am #

      Typically we use mean squared error (MSE) or root mean squared error (RMSE).

  18. Laura May 1, 2019 at 1:18 am #

    This is great, thank you! I’ve been looking for a way to manually calculate prediction from parameters because I need to save parameters for forecasting in another context. And I just couldn’t find any documentation that convinced me how to do it… so thanks for this!

  19. Naveksha Sood May 9, 2019 at 3:31 pm #

    Yes we calculate RMSE but how? What will be the actual and predicted values?

  20. Gunay June 4, 2019 at 4:23 am #

    Hi Jason,

    Firstly, thanks for the tutorial. I have a couple of questions in my mind. Firstly, if I do multi-step ahead forecasting and defining AR and MA terms with some number means that the model looks back that amount out of back steps and do the forecast for the future? If I introduce the whole historical time series to the model, how the model uses that whole historical data to do multi-step ahead forecasting? how the coefficients in the AR and MA terms are updated? I would very thankful If I understand them.

    Kind Regards,
    Gunay

  21. Emilio June 6, 2019 at 6:27 pm #

    Hi Jason,

    Thank you for the tutorial! I have a question about settting yhat. What would yhat equal in the (0,0,0), (0, 1, 1), (1,1,0), and (0,1,0) cases and, for those cases, what parameters should we use for the predict function?

    Thank you,
    Emilio

    • Jason Brownlee June 7, 2019 at 7:53 am #

      I’m not sure I follow.

      yhat is the prediction from the model.

      The order (e.g. (0,1,0)) is the configuration of the model.

      Does that help?

      • Emilio June 10, 2019 at 6:03 pm #

        So say I have a 0,1,0 Arima model. I need to have a constant term to be able to run it. How should I incorporate the constant term into the predict function? For the 0,1,0 case and other cases as well?

Leave a Reply