Getting standard errors from regressions using rpy2

Question

I am using rpy2 for regressions. The returned object has a list that includes coefficients, residuals, fitted values, rank of the fitted model, etc.)

However I can't find the standard errors (nor the R^2) in the fit object. Running lm directly model in R, standard errors are displayed with the summary command, but I can't access them directly in the model's data frame.

How can I get extract this info using rpy2?

Sample python code is

from scipy import random
from numpy import hstack, array, matrix
from rpy2 import robjects 
from rpy2.robjects.packages import importr

def test_regress():
    stats=importr('stats')
    x=random.uniform(0,1,100).reshape([100,1])
    y=1+x+random.uniform(0,1,100).reshape([100,1])
    x_in_r=create_r_matrix(x, x.shape[1])
    y_in_r=create_r_matrix(y, y.shape[1])
    formula=robjects.Formula('y~x')
    env = formula.environment
    env['x']=x_in_r
    env['y']=y_in_r
    fit=stats.lm(formula)
    coeffs=array(fit[0])
    resids=array(fit[1])
    fitted_vals=array(fit[4])
    return(coeffs, resids, fitted_vals) 

def create_r_matrix(py_array, ncols):
    if type(py_array)==type(matrix([1])) or type(py_array)==type(array([1])):
        py_array=py_array.tolist()
    r_vector=robjects.FloatVector(flatten_list(py_array))
    r_matrix=robjects.r['matrix'](r_vector, ncol=ncols)
    return r_matrix

def flatten_list(source):
    return([item for sublist in source for item in sublist])

test_regress()

Answer 1

So this seems to work for me:

def test_regress():
    stats=importr('stats')
    x=random.uniform(0,1,100).reshape([100,1])
    y=1+x+random.uniform(0,1,100).reshape([100,1])
    x_in_r=create_r_matrix(x, x.shape[1])
    y_in_r=create_r_matrix(y, y.shape[1])
    formula=robjects.Formula('y~x')
    env = formula.environment
    env['x']=x_in_r
    env['y']=y_in_r
    fit=stats.lm(formula)
    coeffs=array(fit[0])
    resids=array(fit[1])
    fitted_vals=array(fit[4])
    modsum = base.summary(fit)
    rsquared = array(modsum[7])
    se = array(modsum.rx2('coefficients')[2:4])
    return(coeffs, resids, fitted_vals, rsquared, se) 

Although, as I said, this is literally my first foray into RPy2, so there may be a better way to do that. But this version appears to output arrays containing the R-squared value along with the standard errors.

You can use print(modsum.names) to see the names of the components of the R object (kind of like names(modsum) in R) and then .rx and .rx2 are the equivalent of [ and [[ in R.

Answer 2

@joran: Pretty good. I'd say that it is pretty much the way to do it.

from rpy2 import robjects 
from rpy2.robjects.packages import importr

base = importr('base')
stats = importr('stats') # import only once !

def test_regress():
    x = base.matrix(stats.runif(100), nrow = 100)
    y = (x.ro + base.matrix(stats.runif(100), nrow = 100)).ro + 1 # not so nice
    formula = robjects.Formula('y~x')
    env = formula.environment
    env['x'] = x
    env['y'] = y
    fit = stats.lm(formula)
    coefs = stats.coef(fit)
    resids = stats.residuals(fit)    
    fitted_vals = stats.fitted(fit)
    modsum = base.summary(fit)
    rsquared = modsum.rx2('r.squared')
    se = modsum.rx2('coefficients')[2:4]
    return (coefs, resids, fitted_vals, rsquared, se)