Visualizing an ALU pipeline when operations have different throughput

Question

How can I visualize an ALU pipeline when operations have different throughput?

An ALU pipeline where all operations have the same throughput can be visualized as a factory with a conveyor belt assembly line. The depth and latency of the pipeline becomes the number of stations along the conveyor belt.

But an ALU pipeline often has different throughputs for different types of operations. For instance, a MUL may have half the throughput of an ADD. So, if a MUL is added to the pipeline in the current clock cycle, it might be ok to add an ADD in the next cycle, but not to add a new MUL.

This is kind of like saying that if you have a factory that manufactures two types of cars, you might not be able to have two cars of type A follow each other on the conveyor belt. Though, a car of type A followed by a car of type B might be ok.

Answer 1

It's like having the same team working in two different consecutive stations. If you have two consecutive MUL, the team is still busy from the first one (working at the second station) when the second MUL arrives.

Answer 2

Think of a toll road pay station. You have multiple lanes, some of which are dedicated to trucks, some for buses, and some (most, I guess) for private cars. The time it takes for each one to pay is probably quite different, and the number of incoming vehicles of each type is also different, so the question is - how many booths of each type you put?

The area is limited of course, and let's assume that the designer plans to reduce the average wait time. You still could have multiple incoming vehicles of the same type, that would eventually lead to some of the stations filling up and queues would be formed.

Similarly, in a superscalar CPU, some instructions can be executed in parallel on different ALUs, but in many cases a given group of ALUs may be overstressed, and the operations would have to wait in some queue.

Oh, and to complete the metaphor, let's say out-of-order execution can be imitated by people passing you while honking. Doesn't really add to our description but it fits so well :)