What is the best way to present data from a very large resultset?

Question

I'm writing a report view of an audit trail, and I need to display this in a .jsp. What's the "best" way to get the data from the database to the screen?

We're using Spring for dependency injection, Data Access Objects, and Hibernate. I can use hibernate or straight jdbc for this report.

If I load all the records into memory I run out of memory.

Any ideas that don't involve running the query in the jsp?

Answer 1

It seems like this is a natural place to use pagination of your Hibernate results -- run the query at the Servlet level, and paginate results in a way similar to how this person describes:

http://blog.hibernate.org/Bloggers/Everyone/Year/2004/Month/08/Day/14#pagination

This is the easiest method of implementing Hibernate pagination I've seen...

Answer 2

The Display Tag Library is very good at presenting paginated result sets in servlets or portlets. But it normally works with the whole list loaded into memory. So you will have to do a little work to get it to work with paginated result sets by implementing the org.displaytag.pagination.PaginatedList interface. There is a tutorial on the Display Tag web site. There isn't very much to the tutorial but then again implementing the interface is pretty easy.

Answer 3

Just use paging and only load a certain number of rows on the page at a time.

Answer 4

Another way to do it is to use scroll() to stream one row at the time.