Update an Element if Position is Unknown with Upsert

Question

It looks like you(/I ) cannot have both upsert and an array element update operation.

If you do (python):

findDct = {
  "_id": ObjectId("535e3ab9c36b4417d031402f"),
  'events.ids': '176976332'
}
print col.update(findDct, {"$set" : {"events.$.foo": "bar"} }, upsert=True)

It will throw:

pymongo.errors.DuplicateKeyError: insertDocument :: caused by :: 11000 E11000 
duplicate key error index: test.col.$_id_  dup key: { : ObjectId('535e3ab9c36b4417d031402f') }

This happens because "_id" is of course an index and mongo tries to insert the document as a new since the find query fails on its 'events.ids': '176976332' part (cheat).

Is it possible to update an unknown element in array with upsert True/how?

Answer 1

Yes it is, but you are going about it in the wrong way. Rather than make "finding" the element that you are not sure whether it exists or not, then try to apply the $addToSet operator instead:

db.collection.update(
    { "_id": ObjectId("535e3ab9c36b4417d031402f" },
    {
        "$addToSet": { "events": { "foo": "bar" } }
    },
    { "upsert": true }
)

Please also note from the positional $ operator documentation that you should not use the $ operator with "upserts" as this will result in the field name being interpreted as a "literal" ( which includes the value as in "events.$.foo" ) and that will be the actual field inserted into the document.

Try to make sure that your array "insert/upsert" operations specify the whole array content in order to make this work.

Another adaptation is with the "bulk" methods, the pymongo driver already has a nice API for this, but this is a general form:

db.runCommand({
    "update": "collection",
    "updates": [
        {
            "q": { "_id": ObjectId("535e3ab9c36b4417d031402f" } },
            "u": {
                "$addToSet": { 
                    "events": {
                        "foo": "bar", "bar": "baz"
                    }
                }
            },
            "upsert": true
        },
        {
            "q": { "_id": ObjectId("535e3ab9c36b4417d031402f" } },
            "u": {
                "$set": { "events.foo": "bar" }
            }
        }
    ]
})

But still being very careful that you are not producing duplicates in your sub-document array if you can clearly see the case there. But it is a method, as each update will cascade down even if the first form failed to add anything. Not the best case example, but I hope you see the point.