ForkJoinPool creating thouthands of threads

Question

A simple test that demonstrates the problem:

package com.test;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.ForkJoinTask;
import java.util.concurrent.RecursiveTask;

public class Main extends RecursiveTask<Long> {
    private volatile long start;
    private volatile long end;
    private volatile int deep;

    public Main(long start, long end, int index, int deep) {
        this.start = start;
        this.end = end;
        this.deep = deep;
//        System.out.println(deep + "-" + index);
    }

    @Override
    protected Long compute() {
        long part = (end - start) / 10;
        if (part > 1000 && deep < 10) {
            List<RecursiveTask<Long>> subtasks = new ArrayList<RecursiveTask<Long>>();
            for (int i = 0; i < 10; i++) {
                long subtaskEnd = start + part;
                if (i == 9) {
                    subtaskEnd = end;
                }
                subtasks.add(new Main(start, subtaskEnd, i, deep + 1));
                start = subtaskEnd;
            }

            //CASE 1: generates 3000+ threads
            for (int i = 0; i < 10; i++) {
                subtasks.get(i).fork();
            }

            //CASE 2: generates 4 threads
//            invokeAll(subtasks);

            //CASE 3: generates 4 threads
//            for (int i = 9; i >= 0; i--) {
//                subtasks.get(i).fork();
//            }


            long count = 0;
            for (int i = 0; i < 10; i++) {
                count += subtasks.get(i).join();
            }
            return count;
        } else {
            long startStart = start;
            while (start < end) {
                start += 1;
            }
            return start - startStart;
        }
    }

    private static ForkJoinPool executor = new ForkJoinPool();

    public static void main(String[] args) throws Exception {
        ForkJoinTask<Long> forkJoinTask = executor.submit(new Main(0, Integer.MAX_VALUE / 10, 0, 0));

        Long result = forkJoinTask.get();
        System.out.println("Final result: " + result);
        System.out.println("Number of threads: " + executor.getPoolSize());
    }

}

In this sample I create RecursiveTask that simply counts numbers to create some load on CPU. It devides the incoming range in 10 parts recursivly and when the size of the part is less than 1000 or recursion "deepness" is over 10 it starts to count numbers.

There is 3 cases commented in compute() method. Difference is only in the order of forking subtasks. Depending on the order in which I fork subtasks the number of threads in the end is different. On my system it creates 3000+ threads for the first case and 4 threads for the second and third case.

Question is: what's the difference? Do I really need to know internals of this framework to successfully use it?

Answer 1

This is an old problem that I addressed in an article back in 2011, A Java Fork-Join Calamity The article points to part II which shows the fix? for this in Java8 (substitutes stalls instead of extra threads.)

You really can't do much professionally with this framework. There are other frameworks you can use.