Loading a large and dynamically changing data set into a table

Question

I have a large set of data that occasionally might be updated in the background. As a self teaching experiment I do not wish to rely on any framework. Right now I am trying to find the right place with the MVC/MVVC patterns within a DDD/CQRS context.

In order to populate a table I do a query of a large data set and limit the results to twice the number of displayed rows. Once the user scrolls past half of the query, I query for the next batch of results.

I am loading all results into an array as the user scrolls, just a progressive load without throwing away non displayed rows.

Who is responsible for building the array of results converted to domain objects? The model of the MVC or the data repository?

Ideally I would like to reuse my MVC model without being always concerned about limiting the query results.

If data is updated in the background, a system event is created. However in the cases where a new row was created, I want my application to animate the background insertion of a new row, not just discard the query and reload the table. In such case I guess that a background process must load a copy of the current query and compare it with the existing one. It should create a "New Rows System Event" with the positions of the new rows.

Which component should compare current query results with previous ones in order to calculate insertion and deletion points? A system service? The MVC model?

My final concern is that loading a set of rows with multiple limited queries, might be inconsistent if the data is updated in the background. How/Where should I enforce this consistency?

Answer 1

CQRS and DDD = one Domain model and at least one Query model. Now, it's for your own good to not think of DDD as of something with many rules which broken will make you the DDD public enemy no1.

Let's think a bit: Once a Domain object is updated and a domain event is published, you'll have an event handler which will update the query model. Which query model is exactly the one you need for the UI (the view models).

The client UI should request updates (passing the current model datetime stamp) at an interval. Which updates are also part of the query model. In order to detect what changed, the update should have a datetime stamp which will get compared with the one currently shown. If it's a web app, then I strongly suggest to use a js framework like angularjs.org

Now, The MVC Model in this scenario is the query repository. When insert/edit a row, the Model will be a service (or a command handler) which will update the Domain. Everything related to Domain changes and query model generation is outside the UI, as a part of a Services/Application (call it whatever you wish) layer.