Question

Is there a statsmodels or other Python equivalent for R's step functionality for selecting a formula-based model using AIC?

Was it helpful?

Solution

I really suspect that you are doing the same online course as I do -- the following allows you to get the right answers. If the task at hand is not very computationally heavy (and it isn't in the course), then we can sidestep all the smart details of the step function, and just try all the subsets of the predictors.

For each subset we can calculate AIC as ACI = 2*nvars - 2*result.llf.
And then we just find a subset with the minimal AIC:

import itertools
import numpy as np
import pandas as pd
import statsmodels.api as sm
AICs = {}
for k in range(1,len(predictorcols)+1):
    for variables in itertools.combinations(predictorcols, k):
        predictors = train[list(variables)]
        predictors['Intercept'] = 1
        res = sm.OLS(target, predictors).fit()
        AICs[variables] = 2*(k+1) - 2*res.llf
pd.Series(AICs).idxmin()

OTHER TIPS

The first answer didn't work for me but the below one did. Heavily copied from Kostya.

AICs = {}
for k in range(1,len(predictorcols)+1):
    for variables in itertools.combinations(predictorcols, k):
        predictors = list(variables)
        i = True
        independent =''
        for p in predictors:
            if i:
                independent = p
                i=False
            else:
                independent+='+ {}'.format(p)
        regresion = '$DependentVariable$ ~ {}'.format(independent)
        res = sm.ols(regresion, data=train).fit()
        AICs[variables] = 2*(k+1) - 2*res.llf
pd.Series(AICs).idxmin()
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top