Beginner-level Python threading problems
-
20-08-2019 - |
Question
As someone new to GUI development in Python (with pyGTK), I've just started learning about threading. To test out my skills, I've written a simple little GTK interface with a start/stop button. The goal is that when it is clicked, a thread starts that quickly increments a number in the text box, while keeping the GUI responsive.
I've got the GUI working just fine, but am having problems with the threading. It is probably a simple problem, but my mind is about fried for the day. Below I have pasted first the trackback from the Python interpreter, followed by the code. You can go to http://drop.io/pxgr5id to download it. I'm using bzr for revision control, so if you want to make a modification and re-drop it, please commit the changes. I'm also pasting the code at http://dpaste.com/113388/ because it can have line numbers, and this markdown stuff is giving me a headache.
Update 27 January, 15:52 EST: Slightly updated code can be found here: http://drop.io/threagui/asset/thread-gui-rev3-tar-gz
Traceback
crashsystems@crashsystems-laptop:~/Desktop/thread-gui$ python threadgui.pybtnStartStop clicked
Traceback (most recent call last):
File "threadgui.py", line 39, in on_btnStartStop_clicked
self.thread.stop()
File "threadgui.py", line 20, in stop
self.join()
File "/usr/lib/python2.5/threading.py", line 583, in join
raise RuntimeError("cannot join thread before it is started")
RuntimeError: cannot join thread before it is started
btnStartStop clicked
threadStop = 1
btnStartStop clicked
threadStop = 0
btnStartStop clicked
Traceback (most recent call last):
File "threadgui.py", line 36, in on_btnStartStop_clicked
self.thread.start()
File "/usr/lib/python2.5/threading.py", line 434, in start
raise RuntimeError("thread already started")
RuntimeError: thread already started
btnExit clicked
exit() called
Code
#!/usr/bin/bash
import gtk, threading
class ThreadLooper (threading.Thread):
def __init__ (self, sleep_interval, function, args=[], kwargs={}):
threading.Thread.__init__(self)
self.sleep_interval = sleep_interval
self.function = function
self.args = args
self.kwargs = kwargs
self.finished = threading.Event()
def stop (self):
self.finished.set()
self.join()
def run (self):
while not self.finished.isSet():
self.finished.wait(self.sleep_interval)
self.function(*self.args, **self.kwargs)
class ThreadGUI:
# Define signals
def on_btnStartStop_clicked(self, widget, data=None):
print "btnStartStop clicked"
if(self.threadStop == 0):
self.threadStop = 1
self.thread.start()
else:
self.threadStop = 0
self.thread.stop()
print "threadStop = " + str(self.threadStop)
def on_btnMessageBox_clicked(self, widget, data=None):
print "btnMessageBox clicked"
self.lblMessage.set_text("This is a message!")
self.msgBox.show()
def on_btnExit_clicked(self, widget, data=None):
print "btnExit clicked"
self.exit()
def on_btnOk_clicked(self, widget, data=None):
print "btnOk clicked"
self.msgBox.hide()
def on_mainWindow_destroy(self, widget, data=None):
print "mainWindow destroyed!"
self.exit()
def exit(self):
print "exit() called"
self.threadStop = 1
gtk.main_quit()
def threadLoop(self):
# This will run in a thread
self.txtThreadView.set_text(str(self.threadCount))
print "hello world"
self.threadCount += 1
def __init__(self):
# Connect to the xml GUI file
builder = gtk.Builder()
builder.add_from_file("threadgui.xml")
# Connect to GUI widgets
self.mainWindow = builder.get_object("mainWindow")
self.txtThreadView = builder.get_object("txtThreadView")
self.btnStartStop = builder.get_object("btnStartStop")
self.msgBox = builder.get_object("msgBox")
self.btnMessageBox = builder.get_object("btnMessageBox")
self.btnExit = builder.get_object("btnExit")
self.lblMessage = builder.get_object("lblMessage")
self.btnOk = builder.get_object("btnOk")
# Connect the signals
builder.connect_signals(self)
# This global will be used for signaling the thread to stop.
self.threadStop = 1
# The thread
self.thread = ThreadLooper(0.1, self.threadLoop, (1,0,-1))
self.threadCounter = 0
if __name__ == "__main__":
# Start GUI instance
GUI = ThreadGUI()
GUI.mainWindow.show()
gtk.main()
Solution
Threading with PyGTK is bit tricky if you want to do it right. Basically, you should not update GUI from within any other thread than main thread (common limitation in GUI libs). Usually this is done in PyGTK using mechanism of queued messages (for communication between workers and GUI) which are read periodically using timeout function. Once I had a presentation on my local LUG on this topic, you can grab example code for this presentation from Google Code repository. Have a look at MainWindow
class in forms/frmmain.py
, specially for method _pulse()
and what is done in on_entry_activate()
(thread is started there plus the idle timer is created).
def on_entry_activate(self, entry):
text = entry.get_text().strip()
if text:
store = entry.get_completion().get_model()
if text not in [row[0] for row in store]:
store.append((text, ))
thread = threads.RecommendationsFetcher(text, self.queue)# <- 1
self.idle_timer = gobject.idle_add(self._pulse)# <- 2
tv_results = self.widgets.get_widget('tv_results')
model = tv_results.get_model()
model.clear()
thread.setDaemon(True)# <- 3
progress_update = self.widgets.get_widget('progress_update')
progress_update.show()
thread.start()# <- 4
This way, application updates GUI when is "idle" (by GTK means) causing no freezes.
- 1: create thread
- 2: create idle timer
- 3: daemonize thread so the app can be closed without waiting for thread completion
- 4: start thread
OTHER TIPS
Generally it's better to avoid threads when you can. It's very difficult to write a threaded application correctly, and even more difficult to know you got it right. Since you're writing a GUI application, it's easier for you to visualize how to do so, since you already have to write your application within an asynchronous framework.
The important thing to realize is that a GUI application is doing a whole lot of nothing. It spends most of its time waiting for the OS to tell it that something has happened. You can do a lot of stuff in this idle time as long as you know how to write long-running code so it doesn't block.
You can solve your original problem by using a timeout; telling your GUI framework to call back some function after a delay, and then resetting that delay or starting another delayed call.
Another common question is how to communicate over the network in a GUI application. Network apps are like GUI apps in that they do a whole lot of waiting. Using a network IO framework (like Twisted) makes it easy to have both parts of your application wait cooperatively instead of competitively, and again alleviates the need for extra threads.
Long-running calculations can be written iteratively instead of synchronously, and you can do your processing while the GUI is idle. You can use a generator to do this quite easily in python.
def long_calculation(param, callback):
result = None
while True:
result = calculate_next_part(param, result)
if calculation_is_done(result):
break
else:
yield
callback(result)
Calling long_calculation
will give you a generator object, and calling .next()
on the generator object will run the generator until it reaches either yield
or return
. You would just tell the GUI framework to call long_calculation(some_param, some_callback).next
when it has time, and eventually your callback will be called with the result.
I don't know GTK very well, so I can't tell you which gobject functions you should be calling. With this explanation, though, you should be able to find the necessary functions in the documentation, or at worst, ask on a relevant IRC channel.
Unfortunately there is no good general-case answer. If you clarify with exactly what you're trying to do, it would be easier to explain why you don't need threads in that situation.
You can't restart a stopped thread object; don't try. Instead, create a new instance of the object if you want to restart it after it's truly stopped and joined.
I've played with different tools to help clean up the work with threads, idle processing, etc.
make_idle is a function decorator that allows you to run a task in the background cooperatively. This is a good middle ground between something short enough to run once in the UI thread and not affect the responsiveness of the app and doing a full out thread in special synchronization. Inside the decorated function you use "yield" to hand the processing back over to the GUI so it can remain responsive and the next time the UI is idle it will pick up in your function where you left off. So to get this started you just call idle_add to the decorated function.
def make_idler(func):
"""
Decorator that makes a generator-function into a function that will
continue execution on next call
"""
a = []
@functools.wraps(func)
def decorated_func(*args, **kwds):
if not a:
a.append(func(*args, **kwds))
try:
a[0].next()
return True
except StopIteration:
del a[:]
return False
return decorated_func
If you need to do a bit more processing, you can use a context manager to lock the UI thread whenever needed to help make the code a bit safer
@contextlib.contextmanager
def gtk_critical_section():
gtk.gdk.threads_enter()
try:
yield
finally:
gtk.gdk.threads_leave()
with that you can just
with gtk_critical_section():
... processing ...
I have not finished with it yet, but in combining doing things purely in idle and purely in a thread, I have a decorator (not tested yet so not posted) that you can tell it whether the next section after the yield is to be run in the UI's idle time or in a thread. This would allow one to do some setup in the UI thread, switch to a new thread for doing background stuff, and then switch over to the UI's idle time to do cleanup, minimizing the need for locks.
I haven't looked in detail on your code. But I see two solutions to your problem:
Don't use threads at all. Instead use a timeout, like this:
import gobject
i = 0
def do_print():
global i
print i
i += 1
if i == 10:
main_loop.quit()
return False
return True
main_loop = gobject.MainLoop()
gobject.timeout_add(250, do_print)
main_loop.run()
When using threads, you must make sure that your GUI code is only called from one thread at the same time by guarding it like this:
import threading
import time
import gobject
import gtk
gtk.gdk.threads_init()
def run_thread():
for i in xrange(10):
time.sleep(0.25)
gtk.gdk.threads_enter()
# update the view here
gtk.gdk.threads_leave()
gtk.gdk.threads_enter()
main_loop.quit()
gtk.gdk.threads_leave()
t = threading.Thread(target=run_thread)
t.start()
main_loop = gobject.MainLoop()
main_loop.run()