Regression on stock data using pandas and matplotlib

Question

I'd like to plot what Excel calls an "Exponential Trend/Regression" on a stock chart. When I run the code below in the IPython notebook it simply says "The kernel has died, would you like to restart it?". Any ideas on how to fix it? Also, this is just attempting to do a linear regression and I'm not quite sure how to do a regression on exponential data.

import datetime
import matplotlib.pyplot as plt
import statsmodels.api as sm
from pandas.io.data import DataReader

sp500 = DataReader("AGG", "yahoo", start=datetime.datetime(2000, 1, 1)) # returns a DataFrame
sp500["regression"] = sm.OLS(sp500["Adj Close"], sp500.index).fit().fittedvalues()
top = plt.subplot2grid((3,1), (0, 0), rowspan=2)
top.plot(sp500.index, sp500["Adj Close"], 'b-', sp500.index, sp500["regression"], 'r-')
bottom = plt.subplot2grid((3,1), (2,0))
bottom.bar(sp500.index, sp500.Volume)
plt.gcf().set_size_inches(18,8)

Answer 1

I took another look at this and realized that my previous answer fit poorly since it didn't include an intercept. I've updated my answer.

The segfault comes from trying to us the Datetime index as the exogenous variable. Instead try:

import datetime
import matplotlib.pyplot as plt
import statsmodels.api as sm
import pandas
from pandas.io.data import DataReader

sp500 = DataReader("AGG", "yahoo", start=datetime.datetime(2000, 1, 1)) # returns a DataFrame
sp500["regression"] = sm.OLS(sp500["Adj Close"],
    sm.add_constant(range(len(sp500.index)),
    prepend=True)).fit().fittedvalues

Notice that you don't need to call statsmodels' fittedvalues as a function. If your data points are all equally space this model will give the same results as using the actual index.

For your second question, pandas as a built in exponentially weighted moving average that you may want to look in to: pandas.ewma here.

Answer 2

You're going to need to post more information - version numbers and how you built pandas - maybe a traceback would also help narrow things down. I am unable to replicate a segfault with pandas 0.9.1 on 64-bit linux in IPython with or without pylab. You might also want to report bugs on github issues rather than stack overflow. Easier to get developers' attentions there.