How to run a script for multiple inputs and save result objects based on the name of input?

Question

To run the same set of commands and save the result objects for each time series, I wrote the script in the following manner :

# Specify time series to be used 

dat <- tsname

# Run a set of commands and fit models with different parameters 
dat.1 <- model1(dat)
dat.2 <- model2(dat)
dat.3 <- model3(dat)

# Save objects for further analysis 

tsname.1 <- dat.1
tsname.2 <- dat.2


save(tsname.1, tsname.2, tsname.3, file = paste0("tsname", ".rda")

In this way, we just need to change the script in the beginning and end, save the script for each time series and run each of them individually or in a main script.

The main reason for this method was because I could not find a way to rename the objects created and some search suggested that the above is the only way to do it.

Now as the number of series has increased, it is preferable to either use a for loop, foreach, batch script or commandArgs() to run one script and specify all time series as arguments. To make that work though, the script must find a way to assign these objects with name of series itself so that they can be loaded later for further analysis.

How can we make such a script work or is there a better approach ? Which method of looping will work in that case ?

A MWE

set.seed(1)
tsdata <- ts(rnorm(250), start = c(1980,1), frequency = 12)
dat <- tsdata
dat.11 <- arima(dat, order = c(1, 1, 1))
dat.21 <- arima(dat, order = c(2, 1, 0))

tsname.11 <- dat.11  # problem is to specify this step in each script
tsname.21 <- dat.21
save(tsname.11, , file = "tsname.rda")

REVISED the code

How can we execute this script for multiple time series and store the results and result objects for further analysis ? If Batch command can be used, what is the best way to input set of multiple time series?

How can we run the script for one series, over a set of time series of same or mixed length?

Answer 1

I show a couple ways to create and retrieve individual objects using assign and get, but also provide an alternative where all model runs are stored as different elements of a list. Similarly, I show how you can save each model run in separate files (soi.1.rda, etc), but that you can also save everything together, in one step :)

# ===========================================
# = Set up model params, generate test data =
# ===========================================
mod.param <- 1:5 # orders of AR to try ...
test.soi <- arima.sim(model=list(ar=c(0.5, -0.2)), n=20)

# ===========================================================
# = Create empty vectors/ list to store data and data names =
# ===========================================================
dat.names <- c() # a place to store the names of the individual objects that we'll create
save.names <- c() # the names of the files to save, e.g., "soi.1"
dat.all <- list() # as an altnerative, you can save each analysis in different elements of a list

# ===================================================
# = Loop through each type of model, saving results =
# ===================================================
for(i in 1:length(mod.param)){ # loop through each model you want to run
    temp.dat <- arima(test.soi, order=c(mod.param[i], 0, 0)) # temp.dat is the current model result
    dat.names[i] <- paste("dat", i, sep=".") # dat.names stores the names of all the dat.x objects

    assign(dat.names[i], temp.dat) # use assign() to create an object with name of temp.dat.name

    # dat.all[[dat.names[i]]] <- temp.dat # store the object in a list
    dat.all[[dat.names[i]]] <- get(dat.names[i]) # same as above, but using get(), which complements assign() nicely

    save.name <- paste("soi", i, "rda", sep=".") # I'm assuming the file should be named soi.1.rda, not soi.rda
    save(list=dat.names[i], file=save.name) # save soi.1.rda, soi.2.rda ... etc.
}

# But we don't have to save each file individually!
# We can save a file that contains our list of models (dat.all), as well as each model object (dat.1, dat.2 ... etc.)
all.objs <- ls() # what are all of the object names in our working memory?
dat.objs <- all.objs[all.objs%in%c(dat.names, "dat.all")] # subset to the names of objects we want to save
save(list=dat.objs, file="everything.rda") # save all relevant objects in 1 .rda file


print(dat.1)
print(dat.all$dat.1)

Edit: A different approach that applies each of several models to several time series

Note that the approach might change slightly depending on which models you want to apply to which time series. I've assumed that several models should be applied to each time series, and that the models differ only the the ARIMA order.

The results can be saved as 1 nested list (different model results grouped under different time series), or with model results for each time series being saved as a separate file.

# ============================================================
# = Generate many time series, many sets of model parameters =
# ============================================================
# Model parameters
n.Params <- 5
ar.orders <- 1:n.Params # orders of AR to try ...
i.orders <- rep(0, n.Params)
ma.orders <- rep(0,n.Params)
arima.params <- as.list(as.data.frame(rbind(ar.orders, i.orders, ma.orders)))

# Time Series Data
n.ts <- 10 # number of time series
test.soi <- quote(as.numeric(arima.sim(model=list(ar=c(0.2, 0.4)), n=sample(20:30, 1))))
all.soi.ts <- replicate(n.ts, eval(test.soi))
names(all.soi.ts) <- paste("soi", 1:n.ts, sep=".")

# ==============================================
# = Function to be applied to each time series =
# ==============================================
# Analyze time series
ats <- function(TS, arimaParams){
    dat.all <- list() # as an altnerative, you can save each analysis in different elements of a list
    for(i in 1:length(arimaParams)){ # loop through each model you want to run
        temp.dat <- arima(TS, order=arimaParams[[i]]) # temp.dat is the current model result
        dat.all[[i]] <- temp.dat # store the object in a list
    }
    dat.all 
}

# =========================
# = All Results in 1 List =
# =========================
AllResults <- lapply(all.soi.ts, ats, arima.params) # multilevel list – top level is each TS, within each TS group are the results of all models applied to that time series
save(AllResults, file="everything.rda") # save this big list as 1 file

# ========================================================================
# = Each time series gets its own file and its own list of model results =
# ========================================================================
for(i in 1:length(all.soi.ts)){ # if you want many files, 1 file per time series, use this for loop
    temp.ts <- all.soi.ts[[i]]
    soi.name <- paste("soi", i, sep=".")
    assign(soi.name, ats(temp.ts, arima.params))
    save(list=soi.name, file=paste(soi.name, "rda", sep=".")) # each file will have a name like "soi.1.rda", containing the results of all models applied to the first time series
}

Answer 2

The function sets datname to the name of the input variable. Then define a list L of model outputs and add names. Finally use with(L, ...) to regard the list component names as variable names in ... and use save(list = ..., ...) which allows specification of the variables as a character string of names. Now we only have to set up the data and call the function to run it. If you have several data sets call the function for each one.

run <- function(dat, datname = deparse(subset(dat))) {

  L <- list(
       arima(dat, order = c(1, 1, 1)),
       arima(dat, order = c(2, 1, 0))
     )
  names(L) <- paste(datname, seq_along(L), sep = ".")

  with(L, save(list = names(L), file = paste0(datname, ".rda")))

}

set.seed(1)
soi <- ts(rnorm(250), start = c(1980,1), frequency = 12)
run(soi)

Another possibility might be to save the entire list rather than its components separately. That is, replace the with statement with

  listname <- paste0(datname, ".models")
  assign(listname, L)
  save(list = listname, file = paste0(datname, ".rda"))

REVISED Some corrections and added alternative at end.

Answer 3

When you want to manipulate objects whose names are themselves stored inside a variable, just use assign() and its reverse get(). And use ls() to see which objects exist in a particular scope.

The objects don't need to be stored separately as tsname.1/2/3, model1/2/3??
You can make it real simple if you just store a vector dat[1:3]. Indeed you can have a vector of model[1:3] too. Use vectorization. It's your friend.

Use the assign("tsname.21", object,...) command and its reverse get("tsname.21") to manipulate objects by string name. Just be consistent about whether you prefer to refer to objnames or objects.

set.seed(1)
tsdata <- ts(rnorm(250), start = c(1980,1), frequency = 12)
dat <- tsdata

set.seed(1)
tsdata <- ts(rnorm(250), start = c(1980,1), frequency = 12)
dat <- tsdata

create_model <- function(data, params, objname.prefix='tsname.', envir=.GlobalEnv) {
  objname = paste(objname.prefix, params[1], params[2], sep='') # both assigns and prints it
  the.model <- arima(dat, order = params)
  assign(objname, the.model, envir) # create the var in the global env

  # If you want, you can return the varname 
  return(objname)
}

# dat.11 <- arima(dat, order = c(1, 1, 1))
create_model(dat, c(1, 1, 1))

# dat.21 <- arima(dat, order = c(2, 1, 0))
create_model(dat, c(2, 1, 0))

#tsname.11 <- dat.11  # problem is to specify this step in each script
#tsname.21 <- dat.21
save(tsname.11, , file = "tsname.rda")

# Use `ls(pattern=...)` to find object-names, with wildcard matching.
all.models <- ls(pattern='tsname.*')
#[1] "tsname.11" "tsname.21"

#############
# Refactor your original code similarly.
dat <- tsname

# Run a set of commands and fit models with different parameters 
dat[1] <- model1(dat)
dat[2] <- model2(dat)
dat[3] <- model3(dat)
# or maybe figure out how to use sapply here

# Save objects for further analysis 
tsname <- dat[1:2] # instead of tsname.1 <- dat.1, tsname.2 <- dat.2

#
save(tsname, file = paste0("tsname", ".rda")