The primary problem with your example is that foreach is automatically exporting the entire a
data frame to each of the workers. Instead, try something like:
library(itertools)
foreach(1:10, suba=isplitRows(a, chunkSize=10), .packages='party') %dopar% {
predict(fit, newdata=suba)
}
The 1:10
is for test purposes, to limit the loop to only 10 iterations, as you're doing in your example.
This still requires that fit
be exported to all of the workers, and it might be quite large. But since there are many more tasks than workers and if predict
takes enough time compared to the time to send the test data, it might be worthwhile to parallelize the prediction.