C++11 Thread: Multiple threads waiting on a condition variable

Question 1

std::condition_variable does not specify which waiting thread is woken when you call notify_one. You should therefore write code that doesn't care which thread is woken. The standard pattern is that whichever thread is woken, that thread should do the work that needs to be done.

If you require that the threads are woken in a specific order, then use a different mechanism. You could, for example, have a separate std::condition_variable for each thread, and then put the threads in a queue when they need tools. As a thread hands in the tools, it could then signal the condition variable corresponding to the thread at the front of the queue. That thread will then be woken, and the others will remain sleeping (modulo spurious wake-ups).

Question 2

When there are several threads waiting on a condition, the order in which they are awakened (notify_all) or which one is awakened (notify_one) is unspecified. If you need some sort of ordering, you need to use notify_all, and implement it yourself. You can keep a queue of the threads waiting: before waiting (but after acquiring the mutex), push the thread id onto the end of the queue. In the loop, loop on "this thread at front of queue and necessary tools available". When you get the tools, remove the id from the front of the queue, and call notify_all again.

Question 3

One approach might be to use a fully fledged Semaphore that is shared between the threads instead of a condition variable. That way, you can wait for a specific count.

#include <mutex>
#include <thread>
#include <condition_variable>

using std::mutex;
using std::condition_variable;

class Semaphore
{
public:
    /**
     * Construct a counting semaphore with an initial value
     * @param cnt The value of the initial semaphore count
     */
    Semaphore(unsigned int cnt);

    /**
     * acquire a semaphore count
     * @param numRes The number of count ressources to acquire
     */
    void acquire(unsigned int numRes = 1);

    /**
     * try to acquire a semaphore count.
     * @param numRes The number of count ressources that the method tries to acquire
     * @return true, if numRes could be aquired
     *         false, otherwise
     */
    bool tryAcquire(unsigned int numRes = 1);

    /**
     * release one semaphore cnt
     * @param numRes The number of count ressources to release
     */
    void release(unsigned int numRes = 1);

private:
    unsigned int cnt;
    mutex mut;
    condition_variable cond;
};

Implementation looks like:

void Semaphore::acquire(unsigned int numRes)
{
    unique_lock<mutex> lock(mut);
    while (cnt < numRes)
    {
        cond.wait(lock);
    }

    cnt-=numRes;
}

bool Semaphore::tryAcquire(unsigned int numRes)
{
    unique_lock<mutex> lock(mut);
    if (cnt>=numRes)
    {
        cnt -= numRes;
        return true;
    }
    return false;
}

void Semaphore::release(unsigned int numRes)
{
    {
        unique_lock<mutex> lock(mut);
        cnt += numRes;
    }
    // notify <numRes> waiting entities
    for (unsigned int i = 0; i<numRes; ++i)
    {
        cond.notify_one(); 
    }
}

Question 4

The real issue here is that if you have worker threads and limited amount of needed resources, you should not care which threads actually is activated, you should only care that the work gets done. The only difference here is in the logging. The number of threads that you defined is the number of threads that can run in parallel, which is limited by the resources to one.

If this is not okay for you, then you need to take actions yourself as explained in other answers.