You can take a look at: http://code.google.com/p/vanilla-java/wiki/HugeCollections
Also, have you thought about creating your own implementation of Iterator? Then you can retrieve the needed data on-demand when the iterator performs the next step.
Another option would be to split your dataset - like when you are doing pagination.