Segfault at Py_Finalize (python 2.5) in single threaded C program

Question

In this following hello world C program, I am both extending and embedding Python.

spam.c:

#include <Python.h>

static PyObject *
spam_echo(PyObject *self, PyObject *args) {
    const char *command;
    int sts;

    if (!PyArg_ParseTuple(args, "s", &command))
        return NULL;
    sts = printf("%s\n", command);
    return Py_BuildValue("i", sts);
}

static PyMethodDef SpamMethods[] = {
    {"echo", spam_echo, METH_VARARGS, "Prints passed argument"},
    {NULL, NULL, 0, NULL}
};

PyMODINIT_FUNC
initspam(void) {
    (void) Py_InitModule("spam", SpamMethods);
}

int main(int argc, char *argv[]) {
    PyObject *args;
    PyObject *arg;
    PyObject *result;
    PyObject *moduleName;
    PyObject *module;
    PyObject *func;

    Py_SetProgramName(argv[0]);
    Py_Initialize();
    initspam();

    PyRun_SimpleFile(fopen("foo.py", "r"), "foo.py");

    moduleName = PyString_FromString("__main__");
    module = PyImport_Import(moduleName);
    Py_DECREF(moduleName);
    if (!module) {
        return 1;
    }

    func = PyObject_GetAttrString(module, "foo");
    Py_DECREF(module);
    if (!func || !PyCallable_Check(func)) {
        return 1;
    }

    args = PyTuple_New(1);
    arg = Py_BuildValue("s", "hello world");
    PyTuple_SetItem(args, 0, arg);

    result = PyObject_CallObject(func, args);
    Py_DECREF(arg);
    Py_DECREF(args);
    Py_DECREF(func);

    printf("== before\n");
    Py_Finalize();
    printf("== after\n");
}

And here is the Python program invoked:

foo.py:

#!/usr/bin/python

import spam

def foo(cmd):
    spam.echo(cmd)

I compile with

gcc spam.c -I/usr/include/python2.5/ -lpython2.5

with GCC 4.2.4-1ubuntu4 and I am using python2.5-dev package on Ubuntu Hardy.

Basically, I have a segfault at Py_Finalize as shows the output:

hello world
== before
Segmentation fault

Answer 1

Swapping the lines Py_DECREF(args); and Py_DECREF(arg); solves the problem. The segfault was the result of accessing arg from Py_DECREF(args) after it was already freed.

Answer 2

Perhaps swapping lines fixed it for Python2, but it should not have, and certainly won't for Python 3. Look again at the snippet:

args = PyTuple_New(1);
arg = Py_BuildValue("s", "hello world");
PyTuple_SetItem(args, 0, arg);

PyTuple_New creates a new object, as does Py_BuildValue. If you were to stop there, yes, you should DECREF both.

However, PyTuple_SetItem(args, 0, arg) steals the reference for arg. That means arg is now "owned" by the tuple args. You are no longer are responsible for arg and therefore should not DECREF it.

When args is DECREF, it will DECREF each of its items, which will take care of arg. (Use Py_REFCNT()) to verify, if you like.

That means, for example, if you wanted to put arg into two lists, you'd need to INCREF it once:

args1 = PyTuple_New(1);
args2 = PyTyple_New(1);
arg = Py_BuildValue("s", "hello world");
PyTuple_SetItem(args1, 0, arg);
PyTyple_SetItem(args2, 0, arg);
Py_INCREF(arg);

So when args1 is removed, it can DECREF arg once, without causing a problem with args2.

Note you can INCREF(arg) anywhere after it is created, even after the SetItem -- it's still in scope.