MongoDB query: Using Limit together with $near skips few documents

Question

I am currently developing an app which gets the specific number of documents from a collection if their location cordinates falls within certain range of distance. I am using a active record library for Codeigniter and the query that is generated is as follows

db.updates.find({locs: { $near: [72.844102008984, 19.130207090604 ], $maxDistance: 5000 },  posted_on : { $lt :1398425538.1942 },}).sort( { posted_on: -1 } ).limit(10).toArray()

The problem I am facing is that the above query skips few documents which should actually get pulled. But if I remove the limit(10) from the above query then proper documents gets pulled.

I am not sure, but does using limit() in MongoDB omit few results ? or does it limits to only the closest(nearest) documents?

P.S - The documents skipped using the limit are not always the same & random results are generated

Answer 1

I suspect you are running into problems with the special nature of the $near query. $near performs both a limit() and a sort() on the cursor returning the results -

Specifies a point for which a geospatial query returns the closest documents first. The query sorts the documents from nearest to farthest.

By default, queries that use a 2d index return a limit of 100 documents; however you may use limit() to change the number of results.

http://docs.mongodb.org/manual/reference/operator/query/near/

While the documentation does specifically discuss overriding the limit of 100 with your own limit call

You can further limit the number of results using cursor.limit().

It is silent on adding your own sort() or both sorting and overriding the limit at the same time. I suspect you are running into side effects of doing both. Note that it's not incorrect to do both - it just may not produce the results you are looking for. I'd suggest trying the same query using $geoWithin

http://docs.mongodb.org/manual/reference/operator/query/geoWithin/

$geoWithin does not apply a sort or a limit on the results, so it gives you something of a more raw result set.

Answer 2

Do you have any identical posted_on dates in the system? I recommend sorting by a second key, perhaps _id. If the sort order is non-deterministic the system may skip documents in a non-deterministic manor. Adding the _id field to your sort order is generally not that expensive if you have an index on the other fields as they will already be very close to the correct order and _id is part of all indexes. ("By default, all collections have an index on the _id field, and applications and users may add additional indexes to support important queries and operations." http://docs.mongodb.org/manual/core/index-single/ )