How do I efficiently process multiple results from an Executor Service

Question

I'n new to ExecutorService, but am unsure about my approach to this. I could be dealing with up to 100 threads for a known task. I'm using the general format below, where I create a List of FutureTasks, then submit these to the ExecutorService. The ExecutorService returns and adds these pending results to another list. I then iterate over this list, calling get() on each pending result.

My query is : won't this block on each get() in turn until all 100 threads have completed ? Is there a better way to do this ?

And am I right in assuming that get() returns the result of the Callable implementation's call() method ? I'm using the default FutureTask class, and haven't subclassed it.

ExecutorService exec = Executors.newFixedThreadPool( NUM_THREADS );

List<JobClass> originalList = new ArrayList<JobClass>();

List<SomeOtherClass> pendingResult = new ArrayList<SomeOtherClass>();

List<Future<SomeOtherClass>> resultList = new ArrayList<Future<SomeOtherClass>>();

for( JobClass sc : originalList )
    pendingResult.add( submit( sc );

for( Future<SomeOtherClass> future : futures )
    resultList.add( future.get(5, TimeUnit.SECONDS) ); 

Answer 1

Good question, if I understand you correctly, you are worried about consumption of the result of completed tasks. Yes the thread will block. Java's answer to that is using the CompletionService.

As mentioned in the documentation page "A service that decouples the production of new asynchronous tasks from the consumption of the results of completed tasks".

Answer 2

If you only proceed if all tasks are completed, you can do what you suggested. It will not matter in which order tasks complete. However, if you need to pass task results to other processor as soon as possible or do something else while tasks are executing, you may want to check if task is complete first using isDone method, for example, and call get() if it is complete.