split on factor, sapply, and lm [duplicate]

Question

This question already has answers here:

Linear Regression and group by in R (10 answers)

Closed 5 years ago.

I want to apply lm() to observations grouped by subject, but cannot work out the sapply syntax. At the end, I want a dataframe with 1 row for each subject, and the intercept and slope (ie, rows of: subj, lm$coefficients[1] lm$coefficients[2])

set.seed(1)
subj <- rep(c("a","b","c"), 4) # 4 observations each on 3 experimental subjects
ind <- rnorm(12) #12 random numbers, the independent variable, the x axis
dep <- rnorm(12) + .5 #12 random numbers, the dependent variable, the y axis
df <- data.frame(subj=subj, ind=ind, dep=dep)
s <- (split(df,subj)) # create a list of observations by subject

I can pull a single set of observations from s, make a dataframe, and get what I want:

df2 <- as.data.frame(s[1])
df2
lm1 <- lm(df2$a.dep ~ df2$a.ind)

lm1$coefficients[1]
lm1$coefficients[2]

I am having trouble looping over all the elements of s and getting the data into the final form I want:

lm.list <- sapply(s, FUN= function(x)
  (lm(x[ ,"dep"] ~ x[,"ind"])))
a <-as.data.frame(lm.list)

I feel like I need some kind of transpose of the structure below; the columns (a,b,c) are what I want my rows to be, but t(a) does not work.

head(a)
                                                            a
coefficients                             0.1233519, 0.4610505
residuals        0.4471916, -0.3060402, 0.4460895, -0.5872409
effects          -0.6325478, 0.6332422, 0.5343949, -0.7429069
rank                                                        2
fitted.values 0.74977179, 0.09854505, -0.05843569, 0.47521446
assign                                                   0, 1
                                                             b
coefficients                              1.1220840, 0.2024222
residuals     -0.04461432, 0.02124541, 0.27103003, -0.24766112
effects           -2.0717363, 0.2228309, 0.2902311, -0.2302195
rank                                                         2
fitted.values       1.1012775, 0.8433366, 1.1100777, 1.0887808
assign                                                    0, 1
                                                         c
coefficients                          0.2982019, 0.1900459
residuals     -0.5606330, 1.0491990, 0.3908486, -0.8794147
effects       -0.6742600, 0.2271767, 1.1273566, -1.0345665
rank                                                     2
fitted.values   0.3718773, 0.2193339, 0.5072572, 0.2500516
assign                                                0, 1

Answer 1

By the sounds of it, this might be what you're trying to do:

sapply(s, FUN= function(x)
  lm(x[ ,"dep"] ~ x[,"ind"])$coefficients[c(1, 2)])
#                      a          b          c
# (Intercept) 0.71379430 -0.6817331  0.5717372
# x[, "ind"]  0.07125591  1.1452096 -1.0303726

---

Other alternatives, if this is what you're looking for

I've seen it noted that in general, if you're splitting and then using s/lapply, you can usually just jump straight to by and skip the split step:

do.call(rbind, 
        by(data = df, INDICES=df$subj, FUN=function(x) 
          lm(x[, "dep"] ~ x[, "ind"])$coefficients[c(1, 2)]))
#   (Intercept)  x[, "ind"]
# a   0.7137943  0.07125591
# b  -0.6817331  1.14520962
# c   0.5717372 -1.03037257

Or, you can use one of the packages that lets you do such sorts of calculations more conveniently, like "data.table":

library(data.table)
DT <- data.table(df)
DT[, list(Int = lm(dep ~ ind)$coefficients[1],
          Slo = lm(dep ~ ind)$coefficients[2]), by = subj]
#    subj        Int         Slo
# 1:    a  0.7137943  0.07125591
# 2:    b -0.6817331  1.14520962
# 3:    c  0.5717372 -1.03037257

Answer 2

How about nlme::lmList?

library(nlme)

coef(lmList(dep~ind|subj,df))
##   (Intercept)         ind
## a   0.7137943  0.07125591
## b  -0.6817331  1.14520962
## c   0.5717372 -1.03037257

You can transpose this if you want.