With statement and python threading

Question

I am a noob in python trying to understand the threading module. I am using python 2.7.One of the motivation for with_statement in python was given to be code pattern of

with threading.Lock():
        //User defined function in a new thread

I am not sure if I have understood it correctly but my initial assumption is this code should aquire a lock on mainthread which is released once the child threads are complete. Yet this script

from __future__ import print_function
import threading
import time
import functools
#import contextlib
#Thread module for dealing with lower level thread operations.Thread is limited use Threading instead.

def timeit(fn):
    '''Timeit function like this doesnot work with the thread calls'''
    def wrapper(*args,**kwargs):
        start = time.time()
        fn(*args,**kwargs)
        end = time.time()
        threadID = ""
        print ("Duration for func %s :%d\n"%(fn.__name__ +"_"+ threading.current_thread().name ,end-start))
    return wrapper

exitFlag = 0

@timeit
def print_time(counter,delay):
    while counter:
        if exitFlag:
            thread.exit()
        time.sleep(delay)
        print("%s : %s_%d"%(threading.current_thread().name,time.ctime(time.time()),counter))
        counter -= 1

class Mythread(threading.Thread):
    def __init__(self,threadID,name,counter,delay):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
        self.delay = delay

    def run(self):
        print("Starting%s\n" % self.name)
        print_time(self.counter, self.delay)
        print("Exiting%s\n" % self.name)


if __name__ == '__main__':
    '''
    print_time(5, 1)
     threadLock = threading.Lock()
     threads = []
     thread1 = Mythread(1,"Thread1",5,1)
     thread2 = Mythread(2,"Thread2",5,2)
     thread1.start()
     thread2.start()
     threads.append(thread1)
     threads.append(thread2)
     for t in threads:
         t.join()
    '''
    thread1 = Mythread(1,"Thread1",5,1)
    thread2 = Mythread(2,"Thread2",5,2)
    lock = threading.Lock()
    with lock:
        thread1.start()
        thread2.start()

    print("Exiting main thread ")

Produces following output:

StartingThread1

StartingThread2

Exiting main thread 
Thread1 : Sat May 04 02:21:54 2013_5
Thread1 : Sat May 04 02:21:55 2013_4
Thread2 : Sat May 04 02:21:55 2013_5
Thread1 : Sat May 04 02:21:56 2013_3
Thread1 : Sat May 04 02:21:57 2013_2
Thread2 : Sat May 04 02:21:57 2013_4
Thread1 : Sat May 04 02:21:58 2013_1
Duration for func print_time_Thread1 :5

ExitingThread1

Thread2 : Sat May 04 02:21:59 2013_3
Thread2 : Sat May 04 02:22:01 2013_2
Thread2 : Sat May 04 02:22:03 2013_1
Duration for func print_time_Thread2 :10

ExitingThread2

Please help me understand why the locking doesnot work with the with_statement like this or have I completely misunderstood the concept.I am confused why I get directly to print("Exiting main thread") even by defining a lock

Answer 1

Your existing lock basically does nothing. No other thread has a reference to it, so it can't possibly cause anyone to block anywhere. The only thing it can possibly do is waste a few microseconds. So this:

lock = threading.Lock()
with lock:
    thread1.start()
    thread2.start()

... is pretty much equivalent to:

time.sleep(0.001)
thread1.start()
thread2.start()

And I'm pretty sure that's not what you want.

If you want to force the threads to run sequentially, the easiest way to do that is to just not use threads.

Or, if you must use threads, just wait for one to finish before starting the next:

thread1 = Mythread(1,"Thread1",5,1)
thread2 = Mythread(2,"Thread2",5,2)
thread1.start()
thread1.join()
thread2.start()
thread2.join()

If you want to make the threads serialize themselves, without any help from outside, you have to give them a lock that they can share. For example:

class Mythread(threading.Thread):
    def __init__(self,threadID,name,counter,delay,lock):
        threading.Thread.__init__(self)
        self.lock = lock
        # ...
    def run(self):
        with self.lock:
            # ...

Now, to call them:

lock = threading.Lock()
thread1 = Mythread(1,"Thread1",5,1, lock)
thread2 = Mythread(2,"Thread2",5,2, lock)
thread1.start()
thread2.start()
# ...
thread1.join()
thread2.join()

Now, when each thread starts up, it will try to grab the lock. One will succeed, the other will block until the first one finishes with the lock (by exiting its with statement).

---

If you don't want to serialize the threads, you just want the main thread to wait on all the other threads' completion… all you need for that is join. That's exactly what join is for. There's no need to add anything else.

---

If you really want to, you can daemonize the threads and wait on a sync object instead. I can't think of an easy way to do this with a lock, but it should be pretty easy with a BoundedSemaphore, or a Condition (although you'll have to wait on the condition twice). But this is a very silly thing to do, so I'm not sure why you'd want to.