MongoDB query covered by index but $exists field not contained in the index

Question

I have a collection of the form

{
    "fieldA":"ValueA",
    "fieldB":"ValueB"
}

Where fieldA always exists in the document but fieldB may or may not be in the document.

Lets also assume that I have an index on fieldA called fieldAIndex

Now with this index I would expected the below query to be fully covered by fieldAIndex

db.collection.find({"fieldA":"Value1"},{"_id":0,"fieldA":1})

And running with explain() confirms this to be the case:

{
    "indexOnly" : true
}

However surprisingly when I run the below query:

db.collection.find({"fieldA":"Value1","fieldB":{"$exists":true}},{"_id":0,"fieldA":1})

explain also returns

{
    "indexOnly" : true
}

Given that fieldB is not in the index how can the query return only from the index? Does the index hold information about fields that exists or is the explain returning incorrectly?

Thanks in advance,

Matt.

Answer 1

This behaviour is seen due to a problem in the way indexOnly is reported in explain(). It's due to be fixed in the 2.5.4 mongodb release (https://jira.mongodb.org/browse/SERVER-5759).

To be explicit, checking for the existence of a field not in the index should not return indexOnly=true in the explain(). Any query like this will not be covered by the index.

Thanks to jeffl for pointing me to this mongodb issue.

Answer 2

I believe its the way the Query Engine in MongoDB works. Since your first criteria's project matrix satisfies almost everything it does not need to do anything more.

Here are the indexes on the collection:

[
{
    "v" : 1,
    "key" : {
        "_id" : 1
    },
    "ns" : "test.colTest",
    "name" : "_id_"
},
{
    "v" : 1,
    "key" : {
        "fieldA" : 1
    },
    "ns" : "test.colTest",
    "name" : "fieldA_1"
}
]

Plus you have not done your research correctly. Try adding more documents to the collection and your question will fail.

db.colTest.insert({fieldA:"1"})

Run the query:

db.colTest.find({"fieldA":"value1","fieldB":{"$exists":true}}).explain()

Here is the result

{
"cursor" : "BtreeCursor fieldA_1",
"isMultiKey" : false,
"n" : 1,
"nscannedObjects" : 1,
"nscanned" : 1,
"nscannedObjectsAllPlans" : 1,
"nscannedAllPlans" : 1,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
    "fieldA" : [
        [
            "value1",
            "value1"
        ]
    ]
},
"server" : "local:27017"
}

Additionally, when you query it differently the result of explain() changes again.

db.colTest.find({"fieldB":{"$exists":true}, "fieldA":"value1"}).explain()

Here is the output:

{
"cursor" : "BtreeCursor fieldA_1",
"isMultiKey" : false,
"n" : 1,
"nscannedObjects" : 1,
"nscanned" : 1,
"nscannedObjectsAllPlans" : 1,
"nscannedAllPlans" : 1,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
    "fieldA" : [
        [
            "value1",
            "value1"
        ]
    ]
},
"server" : "local:27017"
}

Please read through documentation, here is the reference: http://docs.mongodb.org/manual/reference/method/cursor.explain/

I think more reliable way of making sure your indexes are being used is to look at "n", "nscanned" and "nscannedObjects". These fields will give you a more clear picture.

Hope this helps!