How to get the queue number from CONDOR into your R job

Question

I think I have a simple problem because I was looking up and down the internet and couldn't find someone else asking this question: My university has a Condor set-up. I want to run several repetitions of the same code (e.g. 100 times). My R code has a routine to store the results in a file, i.e.:

write.csv(res, file=paste(paste(paste(format(Sys.time(), '%y%m%d'),'res', queue, sep="_"), sep='/'),'.csv',sep='',collapse=''))

res are my results (a data.frame), I indicate that this file contains the results with 'res' and finally I want to add the queue number of this calculation (otherwise files would be replaced, wouldn't they?). It should look like: 140109_res_1.csv, 140109_res_2.csv, ...

My submit file to condor looks like this:

universe = vanilla
executable = /usr/bin/R 
arguments =  --vanilla 
log =  testR.log
error = testR.err
input = run_condor.r
output = testR$(Process).txt
requirements = (opsys == "LINUX") && (arch == "X86_64") && (HAS_R_2_13 =?= True)
request_memory = 1000
should_transfer_files = YES
transfer_executable = FALSE
when_to_transfer_output = ON_EXIT
queue 3

I wonder how do I get the 'queue' number into my R code? I tried a simple example with

print(queue)
print(Queue)

But there is no object found called queue or Queue. Any suggestions? Best wishes, Marco

Answer 1

Okay, I solved the problem. This is how it goes:

1. I had to change my submit file. I changed the slot arguments to:

   arguments = --vanilla --args $(Process)

2. Now the process number is forwarded to the R code. There you retrieve it with the following line. The value will be stored as a character. Therefore, you should convert it to a numeric value (also check whether a number like 10 is passed on as '1' and '0' in which case you should also collapse the values).

   run <- commandArgs(TRUE)

Here is an example of the code I let run.

> run <- commandArgs(TRUE)
> run
[1] "0"
> class(run)
[1] "character"
> try(as.numeric(run))
[1] 0
> try(run <- as.numeric(paste(run, collapse='')) )
> try(print(run))
[1] 0
> try(write(run, paste(run,'csv', sep='.')))

You can also find information how to pass on variables/arguments to your code here: http://research.cs.wisc.edu/htcondor/manual/v7.6/condor_submit.html

I hope this helps anyone. Cheers and thanks for all other commenters! Marco