It sounds like your basic strategy is the only known solution to the problem, but I think it can be done more elegantly. For instance, you could avoid creating .BAT files by having each worker execute a different command line based on a worker ID. The worker ID could be assigned using:
# Assign worker ID's to each of the cluster workers
setid <- function(id) assign(".Worker.id", id, pos=globalenv())
clusterApply(cl, seq_along(cl), setid)
Also, you may want to automate the creation of the directories that contain "my.exe". I also prefer to use a symlink rather than a copy of the executable:
# Create directories containing a symlink to the real executable
exepath <- "C:/bin/my.exe" # Path to the real executable
pdir <- getwd() # Parent of the new executable directories
myexe <- file.path(pdir, sprintf("work_%d", seq_along(cl)), "my.exe")
for (x in myexe) {
dir.create(dirname(x), showWarnings=FALSE)
if (file.exists(x))
unlink(x)
file.symlink(exepath, x)
}
If symlinks don't fool "my.exe" into creating the log file in the desired directory, you could try using "file.copy" instead of "file.symlink".
Now you can run your parallel job using:
# Each worker executes a different symlink to the real executable
worker.fun <- function(f, myexe) {
system(sprintf("%s %s /RUN=YES", myexe[.Worker.id], f))
}
ff <- c('a', 'long', 'vector', 'of', 'file', 'paths')
parSapply(cl, ff, worker.fun, myexe)
You could also delete the directories that were created, but they don't use much space since symlinks are used, so it might be better to keep them, especially during debugging/testing.