MongoDB Semantics of undefined vs. null in cursor.sort()

Question

According to information in the MongoDB manpage on cursor.sort (last paragraph on comparing values), NULL values are smaller than any other values in terms of the sort() function.

The question is, are missing attributes regarded as NULL? According to my testing not so:

> db.sort.find().sort({id:1})
{ "_id" : ObjectId("5269554df18e7d2f4bd1241d"), "a" : "Anonym" }
{ "_id" : ObjectId("52695684f18e7d2f4bd12421"), "a" : "Bnonym", "d" : "iii" }
{ "_id" : ObjectId("52695892f18e7d2f4bd12422"), "id" : null, "a" : "Bnonym", "d" : "iii" }
{ "_id" : ObjectId("5269591bf18e7d2f4bd12425"), "id" : null, "a" : "ZZZaaa", "d" : "iii" }
{ "_id" : ObjectId("526954d4f18e7d2f4bd1241b"), "id" : 7, "a" : "Jozo" }

The order seems to be:

• undefined/missing sort attribute
• null value
• the rest according to the order from manpage

Q: The question is, whether there is some undocumented type different from null for missing attributes under the hood?

Answer 1

As far as I can tell (I haven't reviewed the source code specifically), this is undefined and depends on a variety of factors. The most striking one is that reversing the sort does not, in general, reverse the order of null / undefined elements:

> db.sort2.find().sort({"a" : 1});
{ "_id" : ObjectId("526a4d45b4e29833675fb089"), "a" : null }
{ "_id" : ObjectId("526a4d4cb4e29833675fb08b") }
{ "_id" : ObjectId("526a4d48b4e29833675fb08a"), "a" : 21433 }
>
> db.sort2.find().sort({"a" : -1});
{ "_id" : ObjectId("526a4d48b4e29833675fb08a"), "a" : 21433 }
{ "_id" : ObjectId("526a4d45b4e29833675fb089"), "a" : null }
{ "_id" : ObjectId("526a4d4cb4e29833675fb08b") }

Also, when I created an index on a for that test collection, the order changed(!). It didn't change back, however, when I dropped the index. For me, the lack of documentation alone would be enough not to rely on that behavior, but it seems largely unpredictable in even the most simple scenario.

A well-defined and important difference of null and non-existant fields is when using sparse indexes: In a sparse index, null is a valid value and will actually be indexed (it has a value, after all), while a non-existant field is not added to the index at all.