Threaded python and C binding synchronization

Question

I am working on implementing Python bindings for a real-time C library of mine. I have read that threads in Python are not actual threads and that they don't really run in parallel (because of the Global Interpreter Lock). Nevertheless, I would have to consider the following situation.

Imagine the following C function:

int stoppable_lock(mutex *m, bool *stop)
{
    while (!stop || !*stop)
        if (timed_lock(m, SOME_TIME) == SUCCESS)
            return SUCCESS;
    return DIDNT_LOCK;
}

This function waits on a mutex, but through the use of stop, can be cancelled. In the simplest case, to be able to gracefully stop the thread using it, no matter what happens to the other users of the mutex (for example if they die while holding the lock).

My problem is to figure out how to write a Python wrapper for this function. By parsing the arguments (with PyArg_ParseTuple) I can retrieve the mutex, no problem. However, the type of things that can be sent to the function seem to be strings, numbers and objects. The first two obviously cannot replace a bool *, and I doubt writing a wrapper for bool * is a good idea.

My question is, how can I get a parameter, which is a reference to a variable (shared with other python threads) and not just a copy of it?

Here is the function with the piece I'm missing (for a usage like mutex.stoppable_lock(stop)):

static PyObject *_stoppable_lock(_mutex *obj, PyObject *args)
{
    int ret;
    BOOL_STAR stop;
    if (!PyArg_ParseTuple(args, "?", &stop))
    {
        PyErr_SetString(PyExc_ValueError, "Usage: mutex.stoppable_lock(BOOL_STAR)");
        return NULL;
    }
    ret = stoppable_lock(obj->orig, stop);
    if (_set_exception(ret))
        return NULL;
    Py_RETURN_NONE;
}

Where it is unknown to me what BOOL_STAR and "?" should be.

Answer 1

You need to create a wrapper type for bool, with Python mutator functions for the embedded value. If you want to avoid having to create your own type, and if you trust that the C _Bool is the same as C++ bool, then you can use ctypes.c_bool. The Python script could then do

   stop = ctypes.c_bool(False)
   stoppable_lock(mutex, ctypes.addressof(stop))

and the other thread could then do stop.value = True. The extension module would expect a long, and then cast that to a pointer. If you want this more type-safe, you need to create your own wrapper type for bool.

Answer 2

My question is, how can I get a parameter, which is a reference to a variable (shared with other python threads) and not just a copy of it?

You could wrap stop in a mutable object, and pass a reference to that object.

Answer 3

Python threads are real threads (at least in CPython for most thread implementation backends). But you are right, because of the GIL, Python code itself never runs in parallel, although many functions (esp. IO) will do.

Regarding your main question: If you want to directly follow the C implementation logic (having a ref to a bool), NPE already said it: You can wrap it inside a mutable object, like:

class MutableBool:
    value = False

Otherwise, the whole code of stoppable_lock could also easily be reimplemented in Python and then it might become more straight-forward.