Mutexes not working, using queues works. Why?

Question

In this example I'm looking to sync two puts, in a way that the output will be ababab..., without any double as or bs on the output.

I have three examples for that: Using a queue, using mutexes in memory and using mutex with files. The queue example work just fine, but the mutexes don't.

I'm not looking for a working code. I'm looking to understand why using a queue it works, and using mutexes don't. By my understanding, they are supposed to be equivalent.

Queue example: Work.

def a
  Thread.new do
    $queue.pop
    puts "a"
    b
  end
end

def b
   Thread.new do
    sleep(rand)
    puts "b"
    $queue << true
  end
end

$queue = Queue.new
$queue << true
loop{a; sleep(rand)}

Mutex file example: Don't work.

def a
  Thread.new do
    $mutex.flock(File::LOCK_EX)
    puts "a"
    b
  end
end

def b
   Thread.new do
    sleep(rand)
    puts "b"
    $mutex.flock(File::LOCK_UN)
  end
end

MUTEX_FILE_PATH = '/tmp/mutex'
File.open(MUTEX_FILE_PATH, "w") unless File.exists?(MUTEX_FILE_PATH)
$mutex = File.new(MUTEX_FILE_PATH,"r+")
loop{a; sleep(rand)}

Mutex variable example: Don't work.

def a
  Thread.new do
    $mutex.lock
    puts "a"
    b
  end
end

def b
   Thread.new do
    sleep(rand)
    puts "b"
    $mutex.unlock
  end
end

$mutex = Mutex.new
loop{a; sleep(rand)}

Answer 1

Short answer
Your use of the mutex is incorrect. With Queue, you can populate with one thread and then pop it with another, but you cannot lock a Mutex with one one thread and then unlock with another.

As @matt explained, there are several subtle things happening like the mutex getting unlocked automatically and the silent exceptions you don't see.

How Mutexes Are Commonly Used
Mutexes are used to access a particular shared resource, like a variable or a file. The synchronization of variables and files consequently allow multiple threads to be synchronized. Mutexes don't really synchronize threads by themselves.

For example:

1. thread_a and thread_b could be synchronized via a shared boolean variable such as true_a_false_b.
2. You'd have to access, test, and toggle that boolean variable every time you use it - a multistep process.
3. It's necessary to ensure that this multistep process occurs atomically, i.e. is not interrupted. This is when you would use a mutex. A trivialized example follows:

 

require 'thread'
Thread.abort_on_exception = true
true_a_false_b = true
mutex = Mutex.new

thread_a = Thread.new do
  loop do
    mutex.lock
    if true_a_false_b
      puts "a"
      true_a_false_b = false
    end
    mutex.unlock
  end
end

thread_b = Thread.new do
  loop do
    mutex.lock
    if !true_a_false_b
      puts "b"
      true_a_false_b = true
    end
    mutex.unlock
  end

sleep(1) # if in irb/console, yield the "current" thread to thread_a and thread_b

Answer 2

In your mutex example, the thread created in method b sleeps for a while, prints b then tries to unlock the mutex. This isn’t legal, a thread cannot unlock a mutex unless it already holds that lock, and raises an ThreadError if you try:

m = Mutex.new
m.unlock

results in:

release.rb:2:in `unlock': Attempt to unlock a mutex which is not locked (ThreadError)
        from release.rb:2:in `<main>'

You won’t see this in your example because by default Ruby silently ignores exceptions raised in threads other than the main thread. You can change this using Thread::abort_on_exception= – if you add

Thread.abort_on_exception = true

to the top of your file you’ll see something like:

a
b
with-mutex.rb:15:in `unlock': Attempt to unlock a mutex which is not locked (ThreadError)
        from with-mutex.rb:15:in `block in b'

(you might see more than one a, but there’ll only be one b).

In the a method you create threads that acquire a lock, print a, call another method (that creates a new thread and returns straight away) and then terminate. It doesn’t seem to be well documented but when a thread terminates it releases any locks it has, so in this case the lock is released almost immediately allowing other a threads to run.

Overall the lock doesn’t have much effect. It doesn’t prevent the b threads from running at all, and whilst it does prevent two a threads running at the same time, it is released as soon as the thread holding it exits.

I think you might be thinking of semaphores, and whilst the Ruby docs say “Mutex implements a simple semaphore” they are not quite the same.

Ruby doesn’t provide semaphores in the standard library, but it does provide condition variables. (That link goes to the older 2.0.0 docs. The thread standard library is required by default in Ruby 2.1+, and the move seems to have resulted in the current docs not being available. Also be aware that Ruby also has a separate monitor library which (I think) adds the same features (mutexes and condition variables) in a more object-orientated fashion.)

Using condition variables and mutexes you can control the coordination between threads. Uri Agassi’s answer shows one possible way to do that (although I think there’s a race condition with how his solution gets started).

If you look at the source for Queue (again this is a link to 2.0.0 – the thread library has been converted to C in recent versions and the Ruby version is easier to follow) you can see that it is implemented with Mutexes and ConditionVariables. When you call $queue.pop in the a thread in your queue example you end up calling wait on the mutex in the same way as Uri Agassi’s answer calls $cv.wait($mutex) in his method a. Similarly when you call $queue << true in your b thread you end up calling signal on the condition variable in the same way as Uri Agassi’s calls $cv.signal in his b thread.

The main reason your file locking example doesn’t work is that file locking provides a way for multiple processes to coordinate with each other (usually so only one tries to write to a file at the same time) and doesn’t help with coordinating threads within a process. Your file locking code is structured in a similar way to the mutex example so it’s likely it would suffer the same problems.

Answer 3

The problem with file-based version has not been sorted out properly. The reason why it does not work is that f.flock(File::LOCK_EX) does not block if called on the same file f multiple times. This can be checked with this simple sequential program:

require 'thread'

MUTEX_FILE_PATH = '/tmp/mutex'
$fone= File.new( MUTEX_FILE_PATH, "w")
$ftwo= File.open( MUTEX_FILE_PATH)

puts "start"
$fone.flock( File::LOCK_EX)
puts "locked"
$fone.flock( File::LOCK_EX)
puts "so what"
$ftwo.flock( File::LOCK_EX)
puts "dontcare"

which prints everything except dontcare.

So the file-based program does not work because

$mutex.flock(File::LOCK_EX)

never blocks.