Trying to return paged results with Riak using continuation hash - NoSQL pagination

Question

Edit: I added an answer with a more generic approach for NoSQL situations.

I am working on a project using Riak (with LevelDB).

Using the REST API that Riak offers, I am able to get data based on indexes and a range, which returns the results sorted alpha-num by the index, and a continuation hash.

Example call: http://server/buckets/bucketname/index/someindex_int/333333333/555555555?max_results=10&return_terms=true&continuation=somehashhere

Example results:

{
results: [
{
about_river: "12312"
},
{
balloon_tall: "45345"
},
{
basket_written: "23434523"
}
],
continuation: "g2987392479789879087987asdfasdf="
}

I am also making a separate call without specifying max_results and return_terms to get a count of the docs that are in the set. I will know the number of docs per set and the total number of docs, which easily lets us know the number of "pages".

While I am able to make a call for each set of documents based on the hash, then receive a next hash with the results set, I am looking for a way to predict the hashes, therefore pre-populate the client with pagination links.

Is this possible? Are the hashes dynamic based on the index/range info or are they some random value generated by the node your data is returned from?

A coworker has mentioned that the hashes are based on what node you are hitting in the cluster, but I am unable to find documentation on this.

Secondarily, the idea was brought up to cycle through the entire set in the background to get the hashes. This will work, but seems pretty expensive.

I am brand new to Riak and any advice here would be great. I am not able to find any good examples of pagination with Riak. The one that did exist is gone from the internet as far as I can tell.

Answer 1

No, the continuation is not "predictable" nor is anything your co-worker saying correct.

Unfortunately there is no way to know the total number of objects in the range specified except for querying the range without the max_results parameter as you are doing (outside of a 1:1 relation between index key and object key, obviously).

Answer 2

The other answer was the answer I needed, but with some help from CodingHorror, I came up with the answer I wanted.

No pagination. With no pagination, only getting the hash for the next results set is no problem, in fact, it's ideal for my use-case. Just merge that next set onto your existing set(s). But don't let it go on forever.

My inspiration: http://blog.codinghorror.com/the-end-of-pagination/

Thanks, Jeff Atwood!

Answer 3

Ain't the number of results in the response the same?

something like

RiakFuture<SearchOperation.Response, BinaryValue> searchResult = client.executeAsync(searchOp);
searchResult.await();
com.basho.riak.client.core.operations.SearchOperation.Response response = searchResult.get();
logger.debug("number of results {} ", response.numResults());