How do you pass around a void pointer between Python and C when writing an extension?

Question

I started on my first Python extension today and was only creating a very small wrapper around a C library as an exercise. As is typical with C libraries, you start of with an initialization function that yields a handler. You can pass that handler to functions and later you pass it to the cleanup function that frees memory.

When I started writing the wrapper I basically wanted to have a way to call each native C function from python. Quickly I hit the problem that I need to return an arbitrary pointer from C to Python only to give it from there to C again in another function. I doesn't matter how it looks as I don't use it in Python, I just store it and pass it around.

So how do you pass around a void pointer between Python and C?

Please note: I know it is not recommended to write such small wrappers using the extension system but rather ctypes and friends. This is just for practice right now.

Answer 1

PyLong_FromVoidPtr() and PyLong_AsVoidPtr() can be abused to inject malicious data into your program. I recommend against them.

Python has PyCapsule for exactly that job. Capsules provide a safe way to exchange void ptr between modules or Python space and C space. The capsules are type-safe, too. If you need some example, the socket / ssl modules and pyexpat / _elementtree modules use capsules to exchange CAPI structs.

http://docs.python.org/3/c-api/capsule.html

Answer 2

After some searching I found the functions PyLong_AsVoidPtr and PyLong_FromVoidPtr. This yields a nice way to convert between a void * and a PyObject:

# in init function
return PyLong_FromVoidPtr(handle);

# in function using handle
handle = PyLong_AsVoidPtr(python_handle);

The one problem now might be how to retrieve python_handle from the typical *args given to a function:

PyObject *python_handle;
PyArg_ParseTuple(args, "O", &python_handle);

Careful here: The argument given for the "O" object must be a pointer to a PyObject pointer: PyObject **. The "O" itself only denotes to pass this PyObject through without any handling and converting. And with this, you can pass around any pointers any way you like.

Note: I think this solution is not really pretty, because you now have to variables, one that is only needed for a short time.