How to use Rich Domain with Massive Operations?

Question

Since I am working on a relatively complex problem, I would like to use the Domain Driven Design approach to solving it. Problem in question is calculating monthly invoices for clients. The current solution is implemented as a very long Stored Procedure that is difficult to maintain.

I would like to use Object Oriented environment (possibly POCO and Entity Framework) but I am worried about performance. Current SP takes some 10 minutes to generate more than 300 000 records by using set operations. I think this would be very difficult to achieve with any ORM since it will load entity one by one and send updates in the same manner. (Previous version took 5 hours when accessing records one by one.)

How would you create a rich model for massive operations?

Answer 1

I avoid using massive operations as possible as I can when applying rich domain model.

Some batch could be replaced with events. For example, I need a daily order count report.

The batch solution:

A scheduling task which is triggered at the end of the day collects data from orders placed today.

Or use events

The PlaceOrderService publishes a OrderPlacedEvent when a new order is placed. And a eventHandler receives the event and inserts to T_ORDER_COUNT_ENTRY

|TODAY     |ORDER_ID|
|2012-04-01|123     |
|2012-04-01|124     |

The we could use SQL count() to calculate the daily order count report.

Some other batch could run parallelly. For example, My orders should be canceled automatically if unpaid in 30 mins.

The original batch solution is fetch all orders satisfied one by one and invokes their cancel().

The current solution is fetch all orders satisfied one by one and sends a OrderIsOverdue message. Message handlers receive the message containing an orderId and retrieve the order then cancel.

I think this is useful when the cancel operation takes far more time than sends a message . More message handler could be added to improve the thoughput if the hardware resources can afford it.

Answer 2

After giving it a thought, I realized your question is similar to this one:

How to rewrite a stored procedure into the domain-driven code?

Theoretically, it's possible. All you need to do is, identify and separate concerns existing in your stored procedure and rewrite them in the object-oriented manner.

Having said that, I foresee the following kinds of tasks to be solved for this goal:

1 - Data pre-load

Stored procedure can use temporary tables or variables of table type, which are not available in the domain-driven code. Therefore, it comes down to pre-loading the entities. Do it so that you load all the data in advance, without need to load dependent/child entities later when iterating each entity - you need to have them all in the memory.

For this, refer to [aggregate roots]

This approach has a big drawback: high memory consumption. Hence the next step below.

2 - Data iteration without pre-load

Stored procedure has such thing as cursor. That does not load data but instead iterates it in the efficient way. In domain-driven code, you can't really achieve exactly the same effect though. There's something close to cursors - [SqlDatReader], but it does not really use SQL cursors behind the scenes, so be careful with it.

3 - Bulk data modifications

This part can easily be achieved by correctly implementing [Unit of Work] in the domain model. Thus, all the modifications get committed to the database at once, even though you are calling modification operations on the entity instances one-by-one.

I'm not sure if this added some light to the question, but let me know if you have any comments that I could address.