How do I invoke Twisted from a plugin to a GTK program that is already running the main loop?

Question

I wrote a Rhythmbox plugin and I'm trying to add some code to download some JSON asynchronously. Callbacks are registered in the do_activate function:

def do_activate(self):
    shell = self.object
    sp = shell.props.shell_player
    self.db = shell.get_property('db')
    self.qm = RB.RhythmDBQueryModel.new_empty(self.db)
    self.pec_id = sp.connect('playing-song-changed', self.playing_entry_changed)
    self.pc_id = sp.connect('playing-changed', self.playing_changed)
    self.sc_id = sp.connect('playing-source-changed', self.source_changed)
    self.current_entry = None

    ...

I'm trying to download some content when playing_changed is triggered. It currently uses urllib2 to download the content synchronously, but this has the potential to block the UI for a short while. I'd like to use Twisted to solve the problem, but all the examples I've seen use reactor.run(), which blocks indefinitely.

I'm pretty new to Twisted and I was wondering, is there some way to handle this case asynchronously without blocking the main thread?

The full code is here

Answer 1

There isn't any way in twisted to do asynchronous http requests without running IO-loop (reactor.run). Running reactor enables you to use async features not present in python by default. However if your only reason to use twisted is to make async http calls it might be an overkill. Use simple threading instead and make your thread wait for http response.

Answer 2

In the context of a Rhythmbox plugin, you probably need to deal with the fact that the GTK main loop is already running. This is a situation that Twisted supports in principle, but supported APIs to cooperatively initialize a reactor on a main loop that may or may not already have one are tricky.

You can work around it with a function like this:

def maybeInstallReactor():
    import sys
    if 'twisted.internet.reactor' not in sys:
        from twisted.internet import gtk2reactor # s/2/3 if you're using gtk3
        reactor = gtk2reactor.install()
        reactor.startRunning()
        reactor._simulate()
    else:
        from twisted.internet import reactor
    return reactor

Make sure this function is called as early as possible in your program, before anything else gets imported (especially stuff from Twisted).

The startRunning call hooks the reactor up to the GLib main loop, and the _simulate call hooks up Twisted's timed events to a GLib timer.

Sadly, this does involve calling one private function, _simulate, so you'll have to be careful to make sure new versions of Twisted don't break it; but as a result of this question I opened a bug to make this use-case explicitly supported. Plus, beyond this one private method call, nothing else about your usage of Twisted needs to be weird.