Need Help Filling List With For Loop in R

Question

I have a dataset of varying range raw inputs. I'd like to store those as a list in R. The data is 4-Dimensional.

I created a for loop to store the data in a nested list, but the data seems to only fall into the last list. I can't seem to find a problem with my loop.

I have two different lists. In one of those, I store the data stacked together, that works fine so I don't seem to have a problem with data acquision. As for the nested loop, it skips the first layers of lists and stores the last acquired data properly. I attachted the code and I'd appreciate any help. Thanks.

mask1 = list()
mask2 = list()
mask3 = list()
mask4 = list()
mask5 = list()

mask1$vid = list()
mask2$vid = list()
mask3$vid = list()
mask4$vid = list()
mask5$vid = list()

mask1$all = list()
mask2$all = list()
mask3$all = list()
mask4$all = list()
mask5$all = list()

for (i in 1:6766){
  f <- i%%199
  v <- ceiling(i/199)
  if (f==0){
    f=199;
  }

  mask1$vid[[v]] = list()
  mask2$vid[[v]] = list()
  mask3$vid[[v]] = list()
  mask4$vid[[v]] = list()
  mask5$vid[[v]] = list()

  mask1$vid[[v]][[f]] = list()
  mask2$vid[[v]][[f]] = list()
  mask3$vid[[v]][[f]] = list()
  mask4$vid[[v]][[f]] = list()
  mask5$vid[[v]][[f]] = list()

  data1 = read.table(mask1_dir[i])
  data2 = read.table(mask2_dir[i])
  data3 = read.table(mask3_dir[i])
  data4 = read.table(mask4_dir[i])
  data5 = read.table(mask5_dir[i])

  data1=t(data1)
  data2=t(data2)
  data3=t(data3)
  data4=t(data4)
  data5=t(data5)


  for (j in 1:8) {
    m1 <- data1[,j]
    m2 <- data2[,j]
    m3 <- data3[,j]
    m4 <- data4[,j]
    m5 <- data5[,j]

    m1<-m1[m1>0]
    m2<-m2[m2>0]
    m3<-m3[m3>0]
    m4<-m4[m4>0]
    m5<-m5[m5>0]


    mask1$vid[[v]][[f]][[j]] = t(data.frame(m1))
    mask2$vid[[v]][[f]][[j]] = t(data.frame(m2))
    mask3$vid[[v]][[f]][[j]] = t(data.frame(m3))
    mask4$vid[[v]][[f]][[j]] = t(data.frame(m4))
    mask5$vid[[v]][[f]][[j]] = t(data.frame(m5))

    mask1$vid[[v]][[f]][[j]]

    if (v==1 & f==1){
      mask1$all[[j]] = t(data.frame(m1))
      mask2$all[[j]] = t(data.frame(m2))
      mask3$all[[j]] = t(data.frame(m3))
      mask4$all[[j]] = t(data.frame(m4))
      mask5$all[[j]] = t(data.frame(m5))

    }
      else{
        for (k in 1:length(m1)){
          mask1$all[[j]][(length(mask1$all[[j]])+1)] = m1[k]}
        for (k in 1:length(m2)){
          mask2$all[[j]][(length(mask2$all[[j]])+1)] = m2[k]}
        for (k in 1:length(m3)){
          mask3$all[[j]][(length(mask3$all[[j]])+1)] = m3[k]}
        for (k in 1:length(m4)){
          mask4$all[[j]][(length(mask4$all[[j]])+1)] = m4[k]}
        for (k in 1:length(m5)){
          mask5$all[[j]][(length(mask5$all[[j]])+1)] = m5[k]}
    }
  }

}

Answer 1

The main problem you have is that as you look over i in 1:6766 you are re-initializing all the list elements to empty list()s again (ie mask1$vid[[v]] = list() will clear out whatever was in the [[v]] index from an earlier loop). So you need to separate initialization from accumulation. You also have a lot of redundant code.

Here is one possible alternative that uses a list for the 1 MASKs and that uses a 3-D list array for vid rather than a list of lists of lists.

M<-5
F<-199
V<-34
J<-8

#full initialization
mask <- lapply(1:M, function(x) {
    list(
        vid=array(list(NULL),dim=c(V,F,J)), 
        all=lapply(1:J, function(x) numeric(0)))
})
mask_dir<-list(m1=character(F*V), m2=character(F*V), m3=character(F*V), m4=character(F*V),m5=character(F*V))

read.table.fake<-function(...) {return(data.frame(matrix(rep(1:3, each=J), nrow=J, ncol=3)))}

for(f in 1:F) {
    for(v in 1:V) {
        i<- f + (v-1)*199
        data<-lapply(1:M, function(x) t(read.table.fake(mask_dir[[x]][i], i)))
        for(j in 1:J) {
            ppdata <- lapply(1:M, function(x) Filter(function(z) {z>0}, data[[x]][,j]))
            for(m in 1:M) {
                mask[[m]]$vid[[v,f,j]] <- ppdata[[m]]
                mask[[m]]$all[[j]] <- c(mask[[m]]$all[[j]], ppdata[[m]])
            }
        }   
    }
}

I've included a fake read.table because you really didn't specify what your data actually looked like. I assume each file must have 8 rows and perhaps a variable number of columns and all values are numeric? I also combined the mask_dir variables into a list such that mask_dir=list(mask1_dir, mask2_dir, mask3_dir, mask4_dir, mask5_dir)

So the code is shorter but it will take a while to run because that's a lot of looping and file reading.

To get at elements, you could use mask[[5]]$vid[[3,1,2]] rather than mask5$vid[[3]][[1]][[2]] and mask[[2]]$all[[3]] rather than mask2$all[[3]].

To test, i'd start with small values of F and V just to make sure things look good first.