Using poll function with buffered streams

Question

I am trying to implement a client-server type of communication system using the poll function in C. The flow is as follows:

1. Main program forks a sub-process
2. Child process calls the exec function to execute some_binary
3. Parent and child send messages to each other alternately, each message that is sent depends on the last message received.

I tried to implement this using poll, but ran into problems because the child process buffers its output, causing my poll calls to timeout. Here's my code:

int main() {
char *buffer = (char *) malloc(1000);
int n;

pid_t pid; /* pid of child process */

int rpipe[2]; /* pipe used to read from child process */
int wpipe[2]; /* pipe used to write to child process */
pipe(rpipe);
pipe(wpipe);

pid = fork();
if (pid == (pid_t) 0)
{
    /* child */

    dup2(wpipe[0], STDIN_FILENO);
    dup2(rpipe[1], STDOUT_FILENO);
    close(wpipe[0]); close(rpipe[0]);
    close(wpipe[1]); close(rpipe[1]);
    if (execl("./server", "./server", (char *) NULL) == -1)
    {
        fprintf(stderr, "exec failed\n");
        return EXIT_FAILURE;
    }       
    return EXIT_SUCCESS;
}
else
{
    /* parent */

    /* close the other ends */
    close(wpipe[0]);
    close(rpipe[1]);

    /* 
      poll to check if write is good to go 
                This poll succeeds, write goes through
        */
    struct pollfd pfds[1];
    pfds[0].fd = wpipe[1];
    pfds[0].events = POLLIN | POLLOUT;
    int pres = poll(pfds, (nfds_t) 1, 1000);
    if (pres > 0)
    {
        if (pfds[0].revents & POLLOUT)
        {
            printf("Writing data...\n");
            write(wpipe[1], "hello\n", 6);
        }
    }

    /* 
        poll to check if there's something to read.
        This poll times out because the child buffers its stdout stream.
    */
    pfds[0].fd = rpipe[0];
    pfds[0].events = POLLIN | POLLOUT;
    pres = poll(pfds, (nfds_t) 1, 1000);
    if (pres > 0)
    {
        if (pfds[0].revents & POLLIN)
        {
            printf("Reading data...\n");
            int n = read(rpipe[0], buffer, 1000);
            buffer[n] = '\0';
            printf("child says:\n%s\n", buffer);
        }
    }

    kill(pid, SIGTERM);
    return EXIT_SUCCESS;
}
}

The server code is simply:

int main() {
    char *buffer = (char *) malloc(1000);

    while (scanf("%s", buffer) != EOF)
    {
        printf("I received %s\n", buffer);
    }   
    return 0;
}

How do I prevent poll calls from timing out because of buffering?

EDIT:

I would like the program to work even when the execed binary is external, i.e., I have no control over the code - like a unix command, e.g., cat or ls.

Answer 1

You need, as I answered in a related answer to a previous question by you, to implement an event loop; as it name implies, it is looping, so you should code in the parent process:

while (1) { // simplistic event loop!
   int status=0;
   if (waitpid(pid, &status, WNOHANG) == pid)
      { // clean up, child process has ended
        handle_process_end(status);
        break;
      };
   struct pollpfd pfd[2];
   memset (&pfd, 0, sizeof(pfd)); // probably useless but dont harm
   pfd[0].fd = rpipe[0];
   pfd[0].events = POLL_IN;
   pfd[1].fd = wpipe[1];
   pfd[0].event = POLL_OUT;
   #define DELAY 5000 /* 5 seconds */
   if (poll(pfd, 2, DELAY)>0) {
      if (pfd[0].revents & POLL_IN) {
         /* read something from rpipe[0]; detect end of file; 
            you probably need to do some buffering, because you may 
            e.g. read some partial line chunk written by the child, and 
            you could only handle full lines. */
      };
      if (pfd[1].revents & POLL_OUT) {
         /* write something on wpipe[1] */
      };
   }
   fflush(NULL);
} /* end while(1) */

you cannot predict in which order the pipes are readable or writable, and this can happen many times. Of course, a lot of buffering (in the parent process) is involved, I leave the details to you.... You have no influence on the buffering in the child process (some programs detect that their output is or not a terminal with isatty).

What an event polling loop like above gives you is to avoid the deadlock situation where the child process is blocked because its stdout pipe is full, while the parent is blocked writing (to the child's stdin pipe) because the pipe is full: with an event loop, you read as soon as some data is polled readable on the input pipe (i.e. the stdout of the child process), and you write some data as soon as the output pipe is polled writable (i.e. is not full). You cannot predict in advance in which order these events "output of child is readable by parent" and "input of child is writable by parent" happen.

I recommend reading Advanced Linux Programming which has several chapters explaining these issues!

BTW my simplistic event loop is a bit wrong: if the child process terminated and some data remains in its stdout pipe, its reading is not done. You could move the waitpid test after the poll

Also, don't expect that a single write (from the child process) into a pipe would trigger one single read in the parent process. In other words, there is no notion of message length. However, POSIX knows about PIPE_MAX .... See its write documentation. Probably your buffer passed to read and write should be of PIPE_MAX size.

I repeat: you need to call poll inside your event loop and very probably poll will be called several times (because your loop will be repeated many times!), and will report readable or writable pipe ends in an unpredictable (and non-reproducible) order! A first run of your program could report "rpipe[0] readable", you read 324 bytes from it, you repeat the event loop, poll says you "wpipe[1] writable", you can write 10 bytes to it, you repeat the event loop, poll tells that "rpipe[0] readable", you read 110 bytes from it, you repeat the event loop, poll tells again "rpipe[0] readable", you read 4096 bytes from it, etc etc etc... A second run of the same program in the same environment would give different events, like: poll says that "wpipe[1] writable", you write 1000 bytes to it, you repeat the loop, poll says that "rpipe[0] readable, etc....

NB: your issue is not the buffering in the child ("client") program, which we assume you cannot change. So what matters is not the buffered data in it, but the genuine input and output (that is the only thing your parent process can observe; internal child buffering is irrelevant for the parent), i.e. the data that your child program has been able to really read(2) and write(2). And if going thru a pipe(7), such data will become poll(2)-able in the parent process (and your parent process can read or write some of it after POLL_IN or POLL_OUT in the updated revents field after poll). BTW, if you did code the child, don't forget to call fflush at appropriate places inside it.

Answer 2

There seem to be two problems in your code. "stdout" is by default buffered, so the server should flush it explicitly:

printf("I received %s\n", buffer);
fflush(stdout);

And the main program should not register for POLLOUT when trying to read (but you may want register for POLLERR):

pfds[0].fd = rpipe[0];
pfds[0].events = POLLIN | POLLERR;

With these modifications you get the expected output:

$ ./main
Writing data...
Reading data...
child says:
I received hello

Generally, you should also check the return value of poll(), and repeat the call if necessary (e.g. in the case of an interrupted system call or timeout).