Well it's usual to perform an aggregation on a fact table, in which case a distinct would be moot.
Here you need to use count(distinct) to count the number of ID's, but that is what a data warehouse is for. Similarly you might have to run a Sum(duration), or a count(distinct user), or a count(distinct product).
I don't think you have a design problem, you just have to ensure that you have enough available memory for your group by operations to run as far as possible without disk-based sorting. Monitor large queries through V$SQL_WORKAREA_ACTIVE, monitor the SGA and PGA cache advisors, and adjust the memory allocation if required.