The source code comment pretty much explains the functionality of setRepeatLastToken(boolean). But here are the details:
According to this post, Cassandra uses the MD5 or MurMurHash (depending on Cassandra version) algorithm for generating tokens from the keys. Both of these algorithms are fast but can generate collisions (same token value for different keys). Because of that, there might be multiple rows stored under the same token (usually if the data set is large enough).
Cassandra stores the data on nodes based on the tokens. When using the random partitioner, the data retrieval is done in token order (not key order). This makes sense because the records are read from the same node(s) in sequence and generate less traffic than retrieving records from random nodes in the cluster.
When reading from Cassandra using Astyanax with paging, the page (block) size may correspond to the middle of a set of rows with the same token. When the request for the next page comes, Astyanax needs to know whether to start with the next token (an possibly miss the rest of the rows with the same token than didn't fit into the last page) or repeat last token to make sure all the rows from the last key are read (but repeating one or possibly more rows from previous page).
The code comment also warns that if the page size is small enough that only rows with the same token fit into it, the code may enter an infinite loop if setRepeatLastToken is set to true.
I hope this helps anyone else that might be wondering about this feature.