Not getting all the lines from a pipe

Question

I'm working on an assignment in which I need a few processes (parent and children) to communicate. The parent sends file paths to the children, and they have to run the linux file (/usr/bin/file) on them, returning the output to the father.

For now I'm still trying to get one child to work, so for now I assume there's one child.

I'm intend to send multiple file paths to each child (a batch of files), and then read file's output.

The problem:

I use a loop to write a few file paths, but when I read the child's output pipe I don't get all the output I'm supposed to.

The code:

#define Read            0
#define Write           1
#define ParentRead      read_pipe[0]
#define ParentWrite     write_pipe[1]
#define ChildRead       write_pipe[0]
#define ChildWrite      read_pipe[1]
#define PIPE_BUF_LEN    4096

using namespace std;
int main()
{
    /** Pipe for reading for subprocess */
    int read_pipe[2];
    /** Pipe for writing to subprocess */
    int write_pipe[2];
    char buffer[PIPE_BUF_LEN] = "";
    if (pipe(read_pipe) == 0 && pipe(write_pipe) == 0)
    {
        pid_t pid = fork();
        if (pid == -1)
        {
            fprintf(stderr, "Fork failure");
            exit(EXIT_FAILURE);
        }
        else if (pid == 0) //Child process
        {
            close(ParentRead);
            close(ParentWrite);
            dup2 (ChildRead, STDIN_FILENO); /*redirect ChildRead to stdin*/
            dup2 (ChildWrite, STDOUT_FILENO); /*redirect stdout to ChildWrite*/
            char* paramArgs[]={"/usr/bin/file","-n","-f-",NULL};
            execv("/usr/bin/file",paramArgs);
            exit(EXIT_FAILURE);
        }
        else { //Parent process
            close(ChildRead);
            close(ChildWrite);

            for (int i=0; i < 3 ;i++)
            {
                /*write to processes which are ready for writing: */
                fd_set rfds;
                int retval;
                FD_ZERO(&rfds);
                FD_SET(ParentWrite, &rfds);
                retval = select(10, NULL, &rfds, NULL, NULL);
                if (retval == -1)
                {
                    perror("select()");
                }
                else if (retval)
                {
                    write(ParentWrite, "file1\nfile2\n", 12);
                }
                /*read from processes which are ready for reading*/
                FD_ZERO(&rfds);
                FD_SET(ParentRead, &rfds);
                retval = select(10, &rfds, NULL, NULL, NULL);
                if (retval == -1)
                {
                    perror("select()");
                }
                else if (retval)
                {
                    read(ParentRead, buffer, PIPE_BUF_LEN);
                    cout << buffer;
                }
            }
        }
    }
    exit(EXIT_SUCCESS);
}

In this case, I try to run file on "file1\nfile2\n" (notice the -n -f- flags used) in a loop of 3 iteration, expecting to get six lines, but get only three:

file1: ERROR: cannot open `file1' (No such file or directory)

file2: ERROR: cannot open `file2' (No such file or directory)

file1: ERROR: cannot open `file1' (No such file or directory)

When I don't redirect the child's output to the pipe (letting it write to the std_out), I do get all six lines.

Any help will be appreciated.

Answer 1

The scheduler. You are in a loop that asks the child (i.e. the pgm file) to look up two files three times. You only do one read per loop in the parent.

• If the child is scheduled and uninterrupted it performs the first file lookup, writes the pipe, does the second lookup, writes to the pipe. Parent gets scheduled and you read the full output of the child in one read. You are golden (but lucky).
• The child is scheduled, does the first lookup and pipe write and gets interrupted. The parent reads one output. The child gets rescheduled and does the 2nd lookup and writes the pipe. The parent reads one output. Because you only read 3 times you end up only reading 3 of the 6 outputs.
• Variations on the above. Maybe you'll read 1 output then 3 then 1. You just don't know.

Note that the reason you are at least getting a full child output on each read is because pipe writes are guaranteed to be atomic if they under PIPE_BUF length, which these are. If you were using this kind of a routine on, say a socket, you may end up getting some fractional portion of a message on each read.

Solution: You have to read in a loop until you have the number of bytes (or full message) that you expect. In this case I assume the output of file is always one string ending in a newline. But you are asking for two file outputs each time through the loop. So read until that condition is met, namely until you read two full strings.