I never did find a suitable solution for doMC, so for a while I've been doing the following:
library(doParallel)
cl <- makePSOCKcluster(4) # number of cores to use
registerDoParallel(cl)
## computation
stopCluster(cl)
Works every time.
Pergunta
The documentation for doMC seems very sparse, listing only doMC-package and registerDoMC(). The problem I'm encountering is I'll spawn several workers via doMC/foreach, but then when the job is done they just sit there taking up memory. I can go and hunt their process IDs, but I often kill the master process by accident.
library(doMC)
library(foreach)
registerDoMC(32)
foreach(i=1:32) %dopar% foo()
##kill command here?
I've tried following with registerDoSEQ() but it doesn't seem to kill off the processes.
Solução 2
I never did find a suitable solution for doMC, so for a while I've been doing the following:
library(doParallel)
cl <- makePSOCKcluster(4) # number of cores to use
registerDoParallel(cl)
## computation
stopCluster(cl)
Works every time.
Outras dicas
The doMC package is basically a wrapper around the mclapply function, and mclapply forks workers that should exit before it returns. It doesn't use persistent workers like the snow package or the snow-derived functions in the parallel package, so it doesn't need a function like stopCluster to shutdown the workers.
Do you see the same problem when using mclapply directly? Does it work any better when you call registerDoMC with a smaller value for cores?
Are you using doMC from a IDE such as RStudio or R.app on a Mac? If so, you might want try using R from a terminal to see if that makes a difference. There could be a problem calling fork in an IDE.
if you using doParallel
package, and using registerDoParallel(8)
with numbers
you can using unloadNamespace("doParallel")
to kill the multi process
And if you has the name for the clusters you can using stopCluster(cl)
to remove extra workers
By using registerDoSEQ() you simply register the sequential worker, so all parallel workers should stop. This is not a complete solution, but it should work in some cases.