How to save simulation data using for loop in R

Question

I am simulating several data sets using for loop in R and saving the data sets in text files in a folder. Since I need to analyze these data sets, I am importing these data from the folder to R and doing my analyses. I am wondering if there is any way to do both simulation and analysis by keeping them in R as data frames instead of saving and importing. Here is my code:

setwd("C:\\Users\\John\\Desktop\\datageneration")

kitem<-10
N<-100
disc<-rnorm(k,0,1)
diff=rnorm(k,0,1)

irtp<-function(t,a,b,pexp)
{
    pexp<-1/(1+exp(-b*(t-a)))
    pexp
}
for( iter in 1:20) 
{
    X<-mat.or.vec(N,kitem)
    P<-mat.or.vec(N,kitem)
    for(i in 1:N)
    {
        theta<-rnorm(N,0,1)
        assign(paste0("theta", iter), theta)
        filename1 <- paste (" theta",iter ,".txt ", sep ="")
        write.table( get(paste0("theta",iter)) , file = filename1 , row.names =FALSE ,col.names = FALSE )
        for(k in 1:kitem)
        {
            P[i,k]<-irtp(theta[i],diff[k],disc[k],pexp)
            X[i,k]<-ifelse(runif(1)<P[i,k],1,0)
            assign(paste0("X",iter), X)         # HERE'S THE PART THAT I NEED HELP
            filename2 <- paste ("X",iter ,".txt ", sep ="")
            write.table( get(paste0("X",iter)) , file = filename2 , row.names =FALSE ,col.names = FALSE )
        }
    }
}

All I want to do is just to use the generated data files (e.g., theta1, theta2, theta3..., theta20) by calling their names(e.g., theta1). Since I am generating thousands of data sets, I would like to know if I can do it without using write.table then read.table functions. I will be very appreciated if you can help me.

Answer 1

Edited to reflect the need for the X matrices: Create a list of 20 + 20 items with your simulation data, and name the members accordingly:

kitem<-10
N<-100
disc<-rnorm(kitem,0,1)  # not ( k, ... )
diff=rnorm(kitem,0,1)   # not ( k, ... )
pexp <- 1                   # ??? - not needed here

# the list that takes all the produced data
mySim <- as.list( NULL )

# function definition reduced to the necessary
irtp <- function( t, a, b ) {  1 / ( 1 + exp( -b * ( t -a ) ) ) }

for( iter in 1:20 )
{
  # create two matrices to be filled later
  X<-mat.or.vec(N,kitem)
  P<-mat.or.vec(N,kitem)

  # create and name the theta component
  theta = mySim[[ iter ]] <- rnorm( N, 0, 1 )
  names( mySim )[ iter ] <- paste ( "theta", iter, sep ="" )

  # fill and save the matrices
  for( i in 1:N )
  {
    for( k in 1:kitem )
    {
      P[i,k]<-irtp(theta[i],diff[k],disc[k] )  #  don"t need this: ,pexp)
      X[i,k]<-ifelse(runif(1)<P[i,k],1,0)
    }
  }
  mySim[[ 20 + iter ]] <- X
  names( mySim )[ 20 + iter ] <- paste ( "X", iter, sep ="" )
}

You can save the list altogether as an R object, if you want that.

Now you can adress each simulation be name:

head( mySim$theta3 )
[1]  0.96068066  0.01966067 -1.25682531 -0.15128916 -0.75950710 -1.22243883

You can add matrices, dataframes etc. to the list

mySim$tau1 <- c( "lists", "take", "everything" )

You can selectively save list members with the corresponding file name:

filename <- paste( names( mySim )[3], ".txt", sep = "" )
write.table( mySim$theta3, filename )

Is that what you had in mind?

Answer 2

It is not clear what do you want to do , but I think here replicate is what you need.

ss <- replicate(20,replicate(N,rnorm(N,0,1)))

Answer 3

The objects in those Rda files will all have the same name, 'theta', so you will not be able to load them without having any earlier versions overwritten. If you want to save them without names, you can use saveRDS and readRDS giving them varying names. If you want to give them different names before then get save()-ed, then use assign and then save in the manner you are currently using.

All that said, I would be a lot easier to make a list of twenty such objects with varying names and then save them all at once and load them all at once.

N=10
for ( iter in 1:3) 
{
      theta<-rnorm(N,0,1)
    assign(paste0("theta", iter), theta)
    filename1 <- paste (" theta",iter ,".txt ", sep ="")
    write.table( get(paste0("theta",iter)) , file = filename1 , row.names =FALSE ,col.names = FALSE )
}

> ls(patt="theta")
[1] "theta"  "theta1" "theta2" "theta3"