Replacement for setMaximumPoolSize on ForkJoinPool

Question

I noticed that since Scala 2.9.*, the setMaximumPoolSize method appears to have disappeared from ForkJoinPool and it looks like it does what I want. Most discussion of limiting parallelism in Scala's parallel collections centers around the setParallelism method on that same class (which has also disappeared, but isn't as much of a problem for me) but in my case the parallel tasks can perform external IO and will often block on it. The ForkJoinPool then fires up more threads in that case and effectively performs a DOS against the shared IO resource in question, which is not desirable.

Is there any way to limit the number of threads in the pool in some way? I don't really care about the spawn-when-blocked-on-IO behavior but I would like some degree of parallelism since my IO tasks are independent and do not interfere with one another.

Answer 1

The behavior you refer to is called "continuation threads." Since the join() method doesn't do a context switch, the framework will stall without those additional threads. There have been reports of hundred/thousands of these extra threads spawned.

Java8 replaces this awful practice with "continuation threads." That is, the thread continues fetching tasks from the deque, but it can result in a stall or a stack overflow. There is no replacement for code that uses ForkJoinPool.managedBlock() which may be where you are at with the I/O.