rethinkdb: "RqlRuntimeError: Array over size limit" even when using limit()

Question

I'm trying to access a constant number of the latest documents of a table ordered by a "date" key. Note that the date, unfortunately, was implemented (not by me...) such that the value is set as a string, e.g "2014-01-14", or sometimes "2014-01-14 22:22:22". I'm getting a "RqlRuntimeError: Array over size limit 102173" error message when using the following query:

r.db('awesome_db').table("main").orderBy(r.desc("date"))

I tried to overcome this problem by specifying a constant limit, since for now I only need the latest 50:

r.db('awesome_db').table("main").orderBy(r.desc("date")).limit(50)

Which ended with the same error. So, my questions are:

1. How can I get a constant number of the latest documents by date?

2. Is ordering by a string based date field possible? Is this issue has something to do with my first question?

Answer 1

The reason you get an error here is that the orderBy gets evaluated before the limit so it orders the entire table in memory which is over the array limit. The way to fix this is by using and index. Try doing the following:

table.indexCreate("date")
table.indexWait()
table.orderBy({index: r.desc("date")}).limit(50)

That should be equivalent to what you have there but uses an index so it doesn't require loading the entire table into memory.

Answer 2

This code is decision problem.

ro:= r.RunOpts{ArrayLimit: 500000}
r.DB("wrk").Table("log").Run(sessionArray[0],ro)

Answer 3

// This code for Python r.db('awesome_db').table("main").run(sesion, r.runOpts{arrayLimit: 500000})