Huge Data in Listbox(Table)

Question

I want to show a listbox(Table) with nearly 20 Million rows. How can I do so, with lower memory usage and not letting my server die(stop responding) while doing so.

Even you have any Theoretical idea please do share(I will try to implement). Need solution very urgently.

I know I cannot load all the rows at once. I need to ask new rows from server every time I scroll. I have tried it but my scroll is not smooth enough.

Thanks & Regards, Aman

Answer 1

Why not just retrieve the first 100 entries and then once the client scrolls to the bottom you append another 100 entries and so on.

Answer 2

Maybe you could wait for ZK's new feature. Reference http://books.zkoss.org/wiki/Small_Talks/2012/March/Handling_a_Trillion_Data_Using_ZK

Answer 3

You could use http://books.zkoss.org/wiki/ZK Developer's Reference/MVC/View/Renderer/Listbox Renderer.

public void render(Listitem listitem, Object data, int index)

To start, you can implement render in way so that you get element to render from datasource at hand by index from render method. You can use standard cache (if Hibernate is in place) or custom-written one if otherwise (look also at EhCache).

@Erik solution is really fast to implement. To add you could make a button, so that user would be aware that loading more records would cost some time and would think if one really needs to load more. Scrolling can make you Listbox just hang up for a moment.

And always make an upper constraint on maximal number of records you will show at one time - don't pollute your server's memory.

Answer 4

Make paging for the table values and retrieve specific number of records on demand.

Use can use dataTable pluging to make pagination for data records.

Notice that you can retrieve data in synchronous and asynchronous way using this library