You should keep your VHDL at the highest level of abstraction, so don't ever "break it down" as you described. What you are proposing is that you do the synthesis yourself (like creating a carry-lookahead adder) which is a bad idea. You don't know the target device (FPGA or ASIC library) as well as the synthesizer does and you shouldn't try to tell it what to do. If you want to do an addition, use the +
operator and the tools will figure out the best structure that fits your design constraints.
Dividing the design into many modules will often make it more difficult to optimize your design, since optimizations between modules are generally harder to do than optimizations within modules.
Of course, major functional blocks that have well defined interfaces between them should be in separate modules for the sake of maintaining the design and readability. The ALU can be one module, the instruction ROM another, and so forth. These modules have distinct, well-defined functions and there is not much opportunity for intramodule optimization. If you want to get the last possible bit of optimization available, just flatten the design before optimization and let the tools do the work.