Multithreading Architecture for N Repeating Tasks

Question

I have N tasks, each to be repeated after its own specific delay Interval(N) using a fixed thread pool size that will usually be smaller than N.

Since there will usually be a shortage of threads, preference should be given to executing a different task rather than repeating a recently completed task.

I was thinking of using an outer ThreadPoolExecutor with N nested ScheduledThreadPoolExecutors. I'm not really sure how to go about this in the most optimal way since each of those classes maintains its own internal thread pool.

Answer 1

Besides the use of a PriorityQueue as answered by assylias, you can also solve this architecturally, by having a simple executing ThreadPoolExecutor, and another ScheduledExecutorService, which will insert the tasks after a given delay.

So every task has the executive Runnable, and an insertion Runnable, and will, after successful execution, tell the ScheduledExecutorService to run the insertion Runnable after a given delay, which will then put the task back into the ThreadPoolExecutor.

As code:

// myExecutionTask
void run() {
  doSomeWork();
  scheduledExecutor.schedule(myInsertionRunnable, 1000, TimeUnit.MILLISECONDS);
}

and

// myInsertionRunnable
void run () {
  threadPoolExecutor.execute(myExecutionTask);
}

Effectively this will automatically cycle the tasks in the ThreadPoolExecutor, as those tasks that have already been finished, will be at the end of the queue.

Edit: As discussed in comments, when using the scheduler's fixedRate or fixedDelay functionality on a very busy system, tasks added later might be executed less often than task that have been added earlier, as the system seems to prefer tasks that are already executing when deciding for the next one to run.

In contrast my solution above cycles these tasks properly, although there can be no guarantee on a busy system, that the requested delay is exact. So they might be executed later, but at least always in FIFO order.

Answer 2

You could use a PriorityBlockingQueue and use timestamps to define priorities - something like:

class Task {
    AtomicLong lastRun;
    Runnable r;

    void run() {
        r.run();
        lastRun.set(System.currentMillis);
    }
}

Your ScheduledExecutorService (one thread) can then add the task N to a PriorityQueue every Interval(N).

And you can have a separate consumer running in your FixedThreadPool that takes from the Queue (using a reverse comparator so that the tasks run more recently will have a lower priority).

That is a little sketchy but it should work.