How to display large amount of data fast

Question

I have a large database table that I need to display on a Windows Form. The data is some sort of a "category list" which I need to display to the user in a treeview-like structure, there are categories with more sub-categories. Treeview control has delay loading but the problem is there could be hundred thousand root nodes with 4 column string values. I tried adding 100000 nodes to a treeview and it took 5 minutes to complete. Are there any other options for such an operation? Can you give me any ideas? It doesn't have to be a treeview..

Answer 1

I decided to implement a virtual/server mode for the treeview.

Thanks for the answers.

Answer 2

That is a lot of nodes, but did you try calling BeginUpdate() and then EndUpdate() when you added the list of nodes to the treeview? That would probably bring your performance up a bit!

http://msdn.microsoft.com/en-us/library/system.windows.forms.treeview.beginupdate.aspx

For applications like this, I tend to implement features such as "search as you type", which would return category names as they are typed into a textbox. For example, on each keystroke, I would go back to the DB and return the top 10 or so values that begin with what's in the text box. If there are more than 10 results, I either indicate that more results are present, or I tell them to refine their search. IMO direct searching always trumps sorting and/or paging. I hate paging. It's almost always an admission that your search functionality is not good enough!

Answer 3

Perhaps implement some sort of paging mechanism. 100,000 items in a TreeView would be very difficult to read from a user's perspective. Providing only 1,000 or even fewer root nodes at a time would certainly cut back on load times.

Answer 4

You can use cache capabilities. 20000 records generally takes 0,2s to load. Check your language support in order to use it.

regards,

Answer 5

TreeView has an event BeforeExpand. You can use it to determine on the fly which node content to load. That is, you first load only the top level nodes in your TreeView.

If the user is about to expand a node you can fetch the required data and fill the subnodes of that node. Use the Tag property of TreeNode to store an ID which data belongs to which node.

Make sure to use BeginUpdate() and EndUpdate() or use AddRange() instead of Add() to add nodes because it is much faster.