Does increasing the amount of threads make the producer consumer prob faster?

Question

I've implemented something similar to consumer-producer problem using a unbounded linked blocking queue. I have the producer putting objects to the queue and consumers taking it. When I tested the program, doubling the amound of thread each trial while still processing the same amount of objects, the time for all trials seems to be constant. Is it suppose to be constant? Or more thread means faster processing? Not sure if it is my code that is causing the slowness or the synchonization for the shared resource. Any ideas?

Answer 1

It entirely depends on what the bottleneck is:

• If the consumers are processing the elements as fast as the producer is producing them, then adding more consumers won't help
• If the consumers are the bottleneck, then adding more producers will just mean you build up a backlog of work
• If the consumers are all sharing a single resource which is already maxed out (e.g. a saturated network connection or disk) then adding more threads won't help
• If the consumers synchronize on a shared resource which forces them to work in a serial way for a large portion of the time, then adding more threads won't help
• If the consumers are CPU-bound and you've got enough threads to already max out your CPU usage, then adding more threads won't help

You need to look at what's going on while your program is running:

• What does the length of your work queue look like? Does it just keep growing? Is it always close to 0?
• What does your CPU usage look like? What about network / disk usage?

Then analyze what your code is doing, and work out what how parallelizable you expect the problem to be

Answer 2

Probably your synchronization. Assuming your single queue is only consumed one item at a time, blocking other consumers as you consume or produce, then if you want more throughput, you need more queues.

Answer 3

It is a combination, and you have to try to find an near optimal solution. Let's start with a single core system. There you have to find the balance in waiting (typically for IO) and the calculation time. You can saturate both depending on what you are doing. For example if you are limited by disk speed you can not gain anything by adding more threads. Instead you will probably mess up disk scheduling and performance will degrade. On the other hand if you are never waiting for IO, your job is on the other extreme and you will not gain anything by adding more threads on a single core processor.

On a multi-core system you can increase the number of threads to increase performance as long as you don't end up waiting for IO. More cores will not help you there. But as with IO there is an overhead to adding threads so don't go too high.