Piping and forking in python

Question

I'm trying to write a program in python that writes 2 integers into the pipe via the parent function and then reads those same integers through the child function. The child function is then supposed to print out the product of the two.

The problem is,when i run it, it waits the 5 seconds like it's supposed to but then returns a value of 441 instead of 2.

Hope somebody can help figure out how to fix this code :)

import os,time

def child(pipein):
    while True:
        num1 = int(os.read(pipein,32))
        num2 = int(os.read(pipein,32))
        r=(num1)*(num2)
        print(r)
        os._exit(0)

def parent():
    pipein,pipeout = os.pipe()
    x=5
    if os.fork()==0:
        child(pipein)
    else:
        while True:
            num1=str(2)
            num2=str(1)
            line=os.write(pipeout,num1.encode())
            line=os.write(pipeout,num2.encode())
            time.sleep(x)   

parent()

Answer 1

This is what I would make of it;

from __future__ import print_function  #(1)
import os  #(2)
import sys
import time


def child(pipein):  # (3)
    num1 = int(os.read(pipein, 32))  # (6)
    num2 = int(os.read(pipein, 32))
    r = num1 * num2
    print("r = {}".format(r))
    print("Child says bye.") 
    sys.stdout.flush()  # (4)
    os._exit(0)  # (5)


def parent():
    pipein, pipeout = os.pipe()
    x = 1
    if os.fork() == 0:
        print("Launching child")
        child(pipein)
    else:  # (7)
        print("In parent")
        num1 = str(2)  # (8)
        num2 = str(1)
        os.write(pipeout, num1)
        os.write(pipeout, num2)
        print("Parent goes to sleep")
        time.sleep(x)
        print("Parent says bye.")


if __name__ == '__main__':
    parent()

1. This is required to make the print() function work in Python 2.x
2. Style: imports should each be on a separate line
3. No need for a loop if you're going to exit() after the first iteration
4. Otherwise the print might not show up.
5. Style: use two blank lines after a function.
6. Style: use a space after a comma, unless it's at the end of a line.
7. Remove the while loop, or you'll be in an infinite loop.
8. Style: use spaces around operators.

With these modifications I get the following output;

In parent
Launching child
Parent goes to sleep
r = 2
Child says bye.
Parent says bye.

If you want to use multiple processes, it is generally a much better idea to use the multiprocessing module. It has objects for processes and process pools, and communication objects like queues and pipes. Queues are first-in first-out, and they are synchronized; so if you put two items in, you can read two items.

Answer 2

The immediate problem is that your child has an infinite loop, reading num1 over and over forever (or, rather, reading it twice and then blocking forever on a third input that will never come) before doing anything.

Fix that by moving more of the code into the while loop, like this:

def child(pipein):
    while True:
        num1 = int(os.read(pipein,32))
        num2 = int(os.read(pipein,32))
        r=(num1)*(num2)
        print(r)

And you might as well remove the os._exit(0), because you're never going to reach it anyway.

---

Your next problem is that your encoding and decoding steps don't match. You'll get away with this as long as your sys.getdefaultencoding() is something that's a strict superset of ASCII (or, really, as long as its digits match ASCII digits), but you really shouldn't silently rely on that.

---

Next, os.read(pipein,32) can give you the results of a single write, or it can give you the results of up to 32 separate writes combined together. The fact that write (of up to PIPE_BUF) is guaranteed be atomic doesn't help you—it just means you can't end up with the first half of a write and not the second half.

So, most likely, you'll get the 21 in num1, and then 5 seconds later get another 21 in num2, so instead of printing out 2 every 5 seconds, you'll print out 441 every 10 seconds. But even that isn't guaranteed.

Pipes, like TCP sockets, are byte streams, not message streams. Which means you need to build some kind of protocol.

---

There are two really obvious choices here.

First, since you're already reading a fixed record size of (up to) 32 bytes, why not write a fixed record size of exactly 32 bytes? Just change your str lines to something that generates a string of exactly 32 characters that will be encoded to 32 bytes in any encoding that already works, and will parse as a single integer of the appropriate value. Like this:

num1 = format(2, '32d')
num2 = format(1, '32d')

Alternatively, each record can be a pair of whitespace-separated numbers, and records can be separated with newlines. That's trivial to parse. Especially since you're not using non-blocking pipes or anything, so you can just put a file object around them and make it all easy.