ARMAX model forecasting leads to "ValueError: matrices are not aligned" when passing exog values

Question

I'm struggling with forecasting out of sample values with an ARMAX model.

Fitting the model works fine.

armax_mod31 = sm.tsa.ARMA(endog = sales, order = (3,1), exog = media).fit()
armax_mod31.fittedvalues

Forecasting without exogenous values, as far as I have an according model, works fine as well.

arma_mod31 = sm.tsa.ARMA(sales, (3,1)).fit()
all_arma = arma_mod31.forecast(steps = 14, alpha = 0.05)
forecast_arma = Series(res_arma[0], index = pd.date_range(start = "2013-08-21", periods = 14)) 
ci_arma = DataFrame(res_arma[2], columns = ["lower", "upper"])

However as soon as I want to predict out of sample values I run into problems.

all_armax = armax_mod31.forecast(steps = 14, alpha = 0.05, exog = media_out)

leads to "ValueError: matrices are not aligned".

My first idea was, that the length of *media_out* does not fit. I checked it several times and tried out to pass other series as exog. Length of exog is the same as number of steps. I tried out a time series and also only *media_out.values*.

Checked the documentation:

"exog : array
If the model is an ARMAX, you must provide out of sample
values for the exogenous variables. This should not include
the constant."

As far as I understand this is what I do. Any ideas what I'm doing wrong? In addition I found this ipython notebook http://nbviewer.ipython.org/cb6e9b476a41586958b5 while looking for a solution on the web. On In [53]: you can see a similar error. The author's comment suggests a general problem with out-of-sample prediction, am I right?

I'm running python 2.7.3, pandas 0.12.0-1 and statsmodels 0.5.0-1.

Answer 1

Ah, I see the issue. You need to pass in past data too. E.g., if you want to predict 12 steps of an ARMAX(2,q) model then exog should be of length 14. You need the two extra lags to be able to predict 1 step out. So if you ensure exog is 2d, this should work as expected.

I can't see anyway around this, but let me know if you think there's something to improve here. For now I'll note it in the docs.

[Edit: I realized this requirement was stupid. You no longer have to supply any in-sample variables when using ARMA forecast https://github.com/statsmodels/statsmodels/pull/1124.]