Producer/consumer pattern in Java

Question

I'm thinking how to implement the producer/consumer pattern in Java.

Assume that I have 3 threads and a List containing tasks (say it's about 5 tasks). Each thread grab the task from the list and execute it concurrently. My current approach is to use CountDownLatch

int N = 3;
CountDownLatch startSignal = new CountDownLatch(1);
CountDownLatch doneSignal = new CountDownLatch(N);
ConcurrentLinkedQueue<String> tasks = new ConcurrentLinkedQueue<String>();

main() {
    for (int i=0;i<N;i++) {
        new Thread(new Worker()).start();
    }
    startSignal.countDown();
    doneSignal.await();
    System.out.println("done");
}

class Worker implements Runnable {
    public void run() {
        startSignal.await();
            while ((s = tasks.poll()) != null) {
                // do lengthy task here
                if (task failed) {
                    tasks.add(s);
                    return; // assume that task fails badly and have to stop the thread
                }
            }
        doneSignal.countDown();
    }
}

what I wanted to achieve is that if a thread fails when processing a task, it will be added back to the task list to be picked up again by current or any other thread, but with my current approach using CountDownLatch obviously it is not possible to do so because after doneSignal.countDown() is called, the thread assumes it already have finished the task.

What would be the best approach for this scenario? Is using Executor the only way?

Answer 1

I'd say this is an overly complicated (and prone to errors) solution for this case, it would be really simpler to use a common BlockingQueue, a single thread polling from this blocking queue and handing over jobs to an ExecutorService.

Can't see any reason why you would need a CountDownLatch in this case, it just unnecessarily complicates your worker, that has to understand that it is running in a threaded environment and also has to clean up whatever is dirty when it finishes. BlockingQueues and ExecutorServices are there exactly to get you out of these issues.