Using pipe/dup2 to communicate with a Python subprocess

Question

I'd like to use Python to implement a user interface for my C program. However, I can't seem to get communication working. Here's what I've done so far, test.c:

int main()
{
    int pipe_in[2], pipe_out[2];
    if (pipe(pipe_in) != 0 || pipe(pipe_out) != 0)
    {
        perror("pipe");
    return 1;
    }

    int _proc_handle = 0;
    if ((_proc_handle=fork()) == 0)
    {
        printf("Starting up Python interface...\n");
        dup2(pipe_in[0], STDIN_FILENO);
        dup2(pipe_out[1], STDOUT_FILENO);
        close(pipe_in[0]);
        close(pipe_out[1]);
        execlp("python", "python", "interface.py", (char*)NULL);
        perror("execlp");
        printf("Error executing Python.\n");
        exit(1);
    }

    _write_fd = pipe_in[1];
    _read_fd = pipe_out[0];

    sleep(1);
    char buffer[256];
    int n = read(_read_fd, buffer, 11);

    printf("n: %d\n", n);
    printf("buffer: `%s'\n", buffer);
    write(_write_fd, "from C\n", 5);

    return 0;
}

and interface.py is:

import sys
import time

time.sleep(0.1)
print >>sys.stdout, 'from python'
print >>sys.stderr, sys.stdin.readline()

Running this, I expected it to print,

Starting up Python interface...
n: 11
buffer: `from python'
from C

But instead, it simply hangs after,

Starting up Python interface...

Answer 1

Add to your python script:

sys.stdout.flush() # after printing to stdout
sys.stderr.flush() # after printing to stderr

(Line buffering is the default for tty devices, but not for pipes).

In the future you'll want to detect EOF on your pipes in the parent process (and/or in the child), and you'll have to close unused ends of your pipes in parent, too. EDIT: And to close other unused ends of your pipes in a child process.

/* This should be in parent as well */
close(pipe_in[0]);
close(pipe_out[1]);

/* This should be added to the child */
close(pipe_in[1]);
close(pipe_out[0]);