django 1.5 select_for_update considered fragile design

Question

The Django documentation states

If you were relying on “automatic transactions” to provide locking between select_for_update() and a subsequent write operation — an extremely fragile design, but nonetheless possible — you must wrap the relevant code in atomic(). Since Django 1.6.3, executing a query with select_for_update() in autocommit mode will raise a TransactionManagementError.

Why is this considered fragile? I would have thought that this would result in proper transactionality.

Answer 1

Aymeric clarified over email that such a design is fragile because it relies on the implicit transaction boundaries formed by Django 1.5's implicit transactions.

select_for_update(...)
more_code()
save()

This code works in straightforward cases, but if more_code() results in a write operation to the database, then the transaction would close, producing unintended behavior.

Forcing the user to specify the transaction boundaries also leads to clearer code.

Answer 2

select_for_update isn't fragile.

I wrote that "if you were relying on "automatic transactions"" then you need to review your code when you upgrade from 1.5 from 1.6.

If you weren't relying on "automatic transaction", and even more if the concept doesn't ring a bell, then you don't need to do anything.

As pointed out in yuvi's answer (which is very good, thank you!) Django will raise an exception when it encounters invalid code. There's no need to think about this until you see a TransactionManagementError raised by select_for_update.

Answer 3

The answer is just around the corner, in the docs for select_for_update (emphasis mine):

Evaluating a queryset with select_for_update in autocommit mode is an error because the rows are then not locked. If allowed, this would facilitate data corruption, and could easily be caused by calling, outside of any transaction, code that expects to be run in one.

In other words, there's a contradicting behaviour between autocommit and select_for_update, which can cause data corruption. Here's the django developer's discussion where they first proposed solving this issue, to quote (again, emphasis mine):

[...] under Oracle, in autocommit mode, the automatic commit happens immediately after the command is executed -- and so, trying to fetch the results fails for being done in a separate transaction.

However, with any backend, select-for-update in autocommit mode makes very little sense. Even if it doesn't break (as it does on Oracle), it doesn't really lock anything. So, IMO, executing a query that is a select-for-update in autocommit mode is probably en error, and one that is likely to cause data- corruption bugs.

So I'm suggesting we change the behavior of select-for-update queries, to error out [...] This is a backwards-incompatible change [...] These projects should probably be thankful -- they were running with a subtle bug that is now exposed -- but still.

So it was an Oracle-only bug, which shown light over a deeper problem that's relevant for all backends, and so they made the decision to make this an error in django.

Atomic, on the other hand, only commits things to the database after it has verifying that there are no errors, thus solving the issue.