unable to release GIL with Cython when using callback-based C libraries

Question

I'm creating a GTK/GStreamer application (C gstreamer for audio and pygobject gtk for gui) which waits for MIDI input and responds to it. The MIDI capturing is handled with the RTMidi library.

In the past I was able to successfully release the GIL using the nogil and with gil keywords in my Cython .pyx file. The problem is that I need to be able to read MIDI input values from my C callback function from python, so that I can make changes to the GUI based on this input.

Because of this, I restructured my code slightly so that my python code passes a python function object(a function that I want to call in my RTMidi C callback), which I then use Cython to cast to an object of type void* so I can pass this data through C code.

In other words, whenever MIDI input is detected, the python function that I passed as a void* is called. This works fine. The part that isn't working since I added this functionality is the releasing of the GIL. Now, whenever my application starts, my function which accepts MIDI input is not properly started on its own thread, so the GTK gui is never actually shown.

Some info about releasing the GIL in the Cython docs:

Code in the body of the statement must not manipulate Python objects in any way, and must not call anything that manipulates Python objects without first re-acquiring the GIL. Cython currently does not check this.

I do not believe I am violating any of these rules.

Could this be a bug with Cython or am I missing something here?

Has anyone else had any experience using Cython to call python functions from C callbacks?

Here's a reduced version of the .pyx. Note the "with gil" and "nogil" keywords used for releasing and obtaining the GIL:

from libcpp.vector cimport vector

cdef extern from "player.h":
    cdef cppclass Player:
        Player()
        void listen(void(*)(double, vector[unsigned char]*, void*), void* userdata) nogil

cdef extern from "midi.h":
    cdef cppclass MidiInput:
        int midi_listen(void(*)(double, vector[unsigned char]*, void*), void* userdata) nogil

cdef void midi_init(Player* ob, void* function):
    with nogil:
        ob.listen(callback, function)

#helper function which the RTMidi callback is set to
#the python function is called from the callback
cdef void callback(double deltatime, vector[unsigned char]* message, void* userdata) with gil:
    (<object>userdata)()    

cdef class MidiListen:
    cdef MidiInput* thisptr
    def __cinit__(self):
        self.thisptr = new MidiInput()

cdef class PyPlayer:
    cdef Player *thisptr
    def __cinit__(self):
        self.thisptr = new Player()
    def get_midi_in(self, f):
        midi_init(self.thisptr, <void*>f)

and here's the pygobject gtk:

#!/usr/bin/python
from gi.repository import Gtk
import player
import threading

class MyWindow(Gtk.Window):
    def __init__(self):
        Gtk.Window.__init__(self, title="Hello World") 
        self.player = player.PyPlayer()


def test():
    print("hello, world!")

win = MyWindow()

#this way of starting the midi on a separate thread worked before, 
#but not anymore
thread = threading.Thread(target=win.player.get_midi_in(test))
thread.daemon = True
thread.start()

win.connect("delete-event", win.close_app)
win.show_all()
Gtk.main()

Answer 1

Not sure why I didn't catch this before. If you want to pass a function object as a parameter to the thread, do it like so...

thread = threading.Thread(target=win.player.get_midi_in, args=(test,))

Before, I was just calling the function in that line of code and not passing a function object.