MongoDB Atomicity Concerns — Modifying a document in memory

Question

I'm working with MongoDB on my current project and a little confused about the proper way to build support for concurrent modifications.

I have an array of objects. When a request comes in, I want to inspect the last element in that array and make a conditional decision on how to respond. My code looks something like the following:

# Find the last object ID in the array.
last_element_id = str(document['objects'][-1])
if last_element_id != the_element_id_the_request_is_responding_to:
    db.documents.insert({
        ...
    })
else:
    # Append the response to the end of the array.
    document['objects'].append(new_element_id)
    db.documents.save(document)

I'm concerned about the situation in which:

1. In handling request A, I find the last_element_id is valid, and the response should be appended to the end of the list.
2. Before the append() and save() operations complete, another request, B, is handled.
3. B also sees the last_element_id is valid, appends() the response and saves().
4. Now, A's response is enqueued into the end of the array, but the response no longer follows the supposed 'last_element_id' since B's response snuck in beforehand.

What is the proper way to handle this kind of logic in Mongo's atomicity model? I don't want to employ locks if I can avoid them, since the WSGI application may be run in multiple processes simultaneously.

Thanks!

Answer 1

You can use optimistic locking..,, one way is to add a field called version to the document and then increment it on every update, and when you update make sure that the version in the document is the same as the one you read in.

I could give more detail, but this is fairly well documented at http://www.mongodb.org/display/DOCS/Atomic+Operations#AtomicOperations-%22UpdateifCurrent%22

Let me know if this works for you,

Answer 2

You can use findAndModify to atomicallay handle the transaction (atleast this case).

This command accepts upsert flag, which you can use to update or insert the document based on its existence. Check the below example

> db.upsertTest.find()
{ "_id" : 1, "arr" : [ 1 ] }

and issuing this command with the query _id:1 will push (append) new item to arr

> db.upsertTest.findAndModify({query: {_id:1},update:{$push:{arr:2}},upsert:true,"new":true})
{ "_id" : 1, "arr" : [ 1, 2 ] }

and this will create a new one because the _id:2 does not exist already

> db.upsertTest.findAndModify({query: {_id:2},update:{$push:{arr:2}},upsert:true,"new":true})
{ "_id" : 2, "arr" : [ 2 ] }

> db.upsertTest.find()
{ "_id" : 1, "arr" : [ 1, 2 ] }
{ "_id" : 2, "arr" : [ 2 ] }

Hope it helps.