pymongo saving embedded objectIds, InvalidDocumentError

Question

Using the pymongo driver bare to connect python to mongodb, why is it that using an ObjectId instance as the key for an embedded document raises an InvalidDocument error?

I am trying to link documents using objectids and cant seem to understand why I would want to convert them to strings when the ones created automatically for the driver are ObjectId instances.

item = collection.find({'x':'foo'})
item['otherstuff'] = {pymongo.objectid.ObjectId() : 'data about this link'}
collection.update({'x':'foo'}, item)
bson.errors.InvalidDocument: documents must have only string keys, key was ObjectId('4f0b5d4e764df61c67000000')

In practice the linked ids represent documents that contain questions, and the values in the dictionary here keyed as 'otherstuff' for example would represent this individual document's responses to that particular question.

Is there a reason applying objectids like this won't encode into bson and then fails? Is it impossible to nest ObjectIds within documents like this to cross-reference? Have I misunderstood the purpose of them?

Answer 1

The BSON spec dictates that keys must be strings, so PyMongo is right to reject this as an invalid document (and would be regardless of at what level an ObjectId was used as a key, whether at the top level or in an embedded document). This is necessary, among other reasons, so that the query language can be unambiguous. Imagine you had this document (and that it were a valid BSON document):

{ _id: ...,
  "4f0cbe6d7f40d36b24a5c4d7":           true,
  ObjectId("4f0cbe6d7f40d36b24a5c4d7"): false
}

And then you attempted to query with:

db.foo.find({"4f0cbe6d7f40d36b24a5c4d7": false})

Should this return this document? Should that string be auto-boxed into an ObjectId? How would Mongo know when that can be auto-boxed, and how to disambiguate in cases like this document?

A possible alternative solution to your problem is to have an array of embedded documents like:

{ answers: [
    { answer_id: ObjectId("..."), summary: "Good answer to this question" },
    { answer_id: ObjectId("..."), summary: "Bad answer to this question" }
  ]
}

This is valid BSON, and will also be indexable more efficiently. If you add an index on answers, you can search efficiently for exact matches on these subdocuments; if you add an index on answers.answer_id, then you can search efficiently by the ObjectId of the answer you're looking for (and so on).