How does SELECT FOR UPDATE actually work?

Question

I'm trying to implement a queue using MYSQL, and want to make sure I'm understanding SELECT FOR UPDATE correctly.

My table:

Table jobs
Fields: id (INT), state (VARCHAR), queued_time (TIMESTAMP)

When I insert a job, the state is QUEUED. When I lock a job, the state becomes PROCESSING.

I have multiple machines, each using the same DB connection. When a machine is ready to grab something off the queue, it calls

SELECT FROM jobs WHERE state = "QUEUED" ORDER BY queued_time ASC LIMIT 1 FOR UPDATE; UPDATE jobs SET state = "PROCESSING" WHERE state = "QUEUED" ORDER BY queued_time ASC LIMIT 1;

After the query, I check if the UPDATE succeeded, and if it did, I allow the machine to process the job returned by the SELECT FOR UPDATE.

Let's say machines 1 and 2 are ready to grab something off the queue. The queue looks like this:

id        state        queued_time
1         QUEUED       2014-03-30 20:04:43
2         QUEUED       2014-03-30 22:04:43

Machine 1 will execute the SELECT FOR UPDATE at time t1, and the UPDATE at time t2. What happens when machine 2 executes both SELECT FOR UPDATE and the UPDATE between t1 and t2? Which of these happen?

- Machine 1 ends up with job 1, and machine 2's `UPDATE` fails because machine 1 locked the row and never unlocked it (this is my current understanding)

- Machine 2 ends up with job 1, and machine 1's `UPDATE` fails

- Both machines end up with job 1

What happens when machine 2 executes SELECT FOR UPDATE between t1 and t2, and UPDATE after t2? Which of these happen?

- Machine 1 ends up with job 1, and machine 2's `UPDATE` fails

- Machine 2 ends up with job 1, and machine 1's `UPDATE` fails

- Both machines end up with job 1 because machine 2's `UPDATE` succeeds since machine 1 release the lock (this is my current understanding)

Answer 1

You'll need to execute these actions inside of a transaction (autocommit=0) because SELECT FOR UPDATE only holds the lock until the transaction completes.

None of your scenarios occur. The transactions won't immediately fail, but they may time out.

If transaction 2 attempts to lock the same record as transaction 1 (the lock occurs in the SELECT FOR UPDATE), transaction 2 will have to wait until transaction 1 completes. If the lock time out period (innodb_lock_wait_timeout) is reached, then the transaction will time out ("fail").

Ensure that any time you want to select the record and obey any current locks that you use SELECT FOR UPDATE. A normal SELECT without FOR UPDATE won't wait for any locks.

You should, obviously, attempt to finish any transaction as quickly as possible.

Your method should work fine. You should consider retrieving the primary key value in the SELECT FOR UPDATE statement and then simply reference the record by the primary key value in the UPDATE statement.