SWIG: Passing binary data fails

Question

I'm using a C-Library with python bindings generated by swig. I want to pass an arbitrary python string to the C-Library. This string contains unprintable characters, e.g:

\xff\x0A\82

The C-Library uses an char to deal with this data. For converting I use following swig typemap:

%typemap(in) (char *data, int size)
{
        if(!PyUnicode_Check($input))
        {
                PyErr_SetString(PyExc_ValueError, "String value required");
                return NULL;
        }

        $1 = PyBytes_AsString(PyUnicode_AsUTF8String($input));
        $2 = PyUnicode_GetSize($input);
}

This partly works. Only values between 0x00 and 0x7F work. The value 0xFF gets converted to a wrong value namely 0xC3.

Has anyone suggestions to get this work for 0x00 to 0xFF?

Answer 1

The Unicode string '\xff' is converted into the UTF-8 byte sequence b'\xc3\xbf' exactly as you have coded.

If you want to send byte data, use a byte string (b'\xff\x0a\x82') and use a PyBytes method to convert it to a char* and size:

%typemap(in) (char* data,int size) (Py_ssize_t len) %{
    if(PyBytes_AsStringAndSize($input,&$1,&len) == -1)
        return NULL;
    $2 = (int)len;
%}

Note that this should work on 32- and 64-bit Windows and null bytes can be included in the data, since the size is explicitly extracted and it handles the possibility of 64-bit Py_ssize_t.