How to process a long list of data from database with Message-driven bean due to the asynchronous requirement properly

softwareengineering.stackexchange https://softwareengineering.stackexchange.com/questions/413085

Question

There is a task in our JavaEE project: a user in the frontend page A can click a button to request to get a zip with the processed data. As the process of the request in the backend needs plenty of time, this process should be implemented in an asynchronous manner, say after clicking the request button, the user does not need to wait for the request but can go to other frontend page to do other requests and when the zip request processing is complete, there will be a link to the generated zip on page A.

In the backend there is the following workflow:

  1. query a long list of data from the relational database
  2. process the whole list of data, each of which will be mutated and returned as the output
  3. the output will be writen to multiple text files
  4. all the text files will be compressed to a zip file as the final result

Due to the asynchronous requirement, message-drive bean (MDB) is selected as the solution. But how to implement it with MDB?

Current design uses 2 MDB on one server: the first one, DataQueryBean, query data from the database and then send data one-by-one to another MDB, RecordProcessBean, which will process each of the incoming data from DataQueryBean and will store the mutated result into a temporay database table. When all data is processed by SingleRecordProcessBean, the DataQueryBean will send a final command message to RecordProcessBean then the RecordProcessBean will generate the zip.

This design seems to break the single responsibility rule. RecordProcessBean not only process the original incoming data but also generate the final zip, i.e. multiple reasons to change on the RecordProcessBean. Secondly, the times of calling the RecordProcessBean is the amount of data records queried, so I wonder if calling on the same MDB thousands/millions of times would affect the performance even though the calling happens on the same server Would like to hear some experts' opinions about the pros and cons of the current design

It just came across my mind that why not let ONLY ONE MDB, DataProcessBean, query the data and then process all the data in a loop, and then generate the zip in another MDB, ZipBean. Whould like to hear some experts' opinions on this pros and cons of this design candicate as well

Any better design suggestion is welcomed

No correct solution

Licensed under: CC-BY-SA with attribution
scroll top