STL containers may not be thread-safe but I haven't ever hit one that cannot be used at different times on different threads. Passing the ownership to another thread seems safe.
I have used the following a couple of times, so I know it works:
- Create a pointer to a std::vector.
- Create a mutex lock to protect the vector pointer.
- Use new[] to create a std::vector and then reserve() a large size for it.
In the receiver thread:
- Lock the mutex whenever adding an item to the queue. This should be a short lock.
- Add queue item.
- Release the lock.
- If you feel like it signal a condition variable. I sometimes don't: it depends on the design. If the volume is very high and there is no pause in the receive side just skip the condition and poll instead.
On the consumer thread (the disk writer):
- Go look for work to do by polling or waiting on a condition variable:
- Lock the queue mutex.
- Look at the queue length.
- If there is work in the queue assign the pointer to a variable in the consumer thread.
- Use new[] and reserve() to create a new queue vector and assign it to the queue pointer.
- Unlock the mutex.
- Go off and write your items to disk.
- delete[] the used-up queue vector.
Now, depending on your problem you may end up needing a way to block. For example in one of my programs if the queue length ever hits 100,000 items, the producing thread just starts doing 1 second sleeps and complaining a lot. It is one of those things that shouldn't happen, yet does, so you should consider it. Without any limits at all it will just use all the memory on the machine and then crash with an exception, get killed by OOM or just come to a halt in a swap storm.