flock(1) is failing to release lock

Question

I have a set of shell scripts that perform backups on my system using truecrypt and rsync. I want to avoid any possibility of these various scripts running at the same time.

They use flock in a subshell, as described in the manpage:

(
    flock -n 9 || exit 1
    # ... commands executed under lock ...
) 9>/var/lock/mylockfile

However, the lock is always failing to acquire for subsequent runs (with exit status 1).

Yet fuser /var/lock/mylockfile and lsof /var/lock/mylockfile show nothing.

Now, if I add the command flock -u to manually unlock, like this:

(
    flock -n 9 || exit 1
    # ... commands executed under lock ...
    flock -u 9
) 9>/var/lock/mylockfile

Then the scripts work.

Am I using flock correctly? Is it safe to call flock -u inside the block?

FYI The troublesome scripts are the ones that call truecrypt within the command block.

Answer 1

The problem is that you're running TrueCrypt in the background, keeping the fd open. You should close the fd to prevent background processes from hanging on to the lock.

In lieu of your actual code, here's a test case:

foo() {
  ( 
    flock -n 9 && echo "ok" || { echo failed; exit 1; }
    sleep 10 &
  ) 9> lock
}
foo; foo

# Output:
# ok
# failed

sleep is forked with fd 9 open, causing the lock to be kept. Let's close fd 9 when backgrounding the process:

foo() {
  ( 
    flock -n 9 && echo "ok" || { echo failed; exit 1; }
    sleep 10 9>&- & 
  #          ^-------- Right here
  ) 9> lock
}
foo; foo

# Output:
# ok
# ok

Answer 2

Lock files are persistent and dealing with all possible race conditions will transform your script in a complex shell program.

Take a look to FLoM tool: http://sourceforge.net/projects/flom/

with something like:

flom -- my_first_command &
flom -- my_second_command &

you can serialize them without all the dark side effects of persistent file based locks. Here are some relevant use case examples: http://sourceforge.net/p/flom/wiki/FLoM%20by%20examples/

Cheers

Ch.F.