Sporadic segfault in c++ python extension

Question

I have a python object which accesses and downloads some text via HTTP. I'm running this python object, and processing that text, using a c++ code. I.e.

/* CPPCode.cxx */
int main(...) {
    for(int i = 0; i < numURLs; i++) {
        // Python method returns a string
        PyObject *pyValue = PyObject_CallMethod(pyObjectInstance, pyFunctionName, par1, par2....);
        string valString = PyString_AsString(pHistValue);   
        // ... process string ... 
    }
} 

/* PyObject.py */
class PyClass:
    def PyFunction(...):
        try: urlSock = urllib.urlopen(urlName)
        except ...

        while(...) :
             dataStr = urlSock.readline()
             # do some basic string processing....

        return dataStr

Most URLs work fine---the c++ code gets the proper string, I can process it, all is happy and well. A few particular URLs which look (basically) the same as the others on a browser, lead to a segfault in the PyString_AsString() method:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x00000000000000b2
0x000000010007716d in PyString_AsString ()

If I print out the string that should be returned by the python method ('dataStr' in the pseudo-code above), it looks fine! I have no idea what could be causing this problem---any tips on how to procede would be appreciated! Thanks

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

SOLUTION:

The template code I was using had a call to

Py_DECREF(pyValue)

before I called

PyString_AsString(pyValue)

Why it was being deallocated for certain particular function calls, I have no idea. As 'Gecco' says in the comments below,

'PyString_AsString documentation says: "The pointer refers to the internal buffer of string, not a copy. The data must not be modified in any way, unless the string was just created using PyString_FromStringAndSize(NULL, size). It must not be deallocated." '

Answer 1

PyString_AsString documentation says: "The pointer refers to the internal buffer of string, not a copy. The data must not be modified in any way, unless the string was just created using PyString_FromStringAndSize(NULL, size). It must not be deallocated."

Please ensure you do not deallocate this buffer

Answer 2

If you compile your C code with the -g debug flag (in GCC at least) then you can run your python code using the gnu debugger gdb:

$ gdb /path/to/python/compiled/against 
... blah ...
(gdb) run PyObject.py

and you should catch your segfault.

Answer 3

My guess is the Py_DECREF is somehow getting a NULL value.