First program in bash pipeline is blocking

StackOverflow https://stackoverflow.com/questions/23188483

  •  06-07-2023
  •  | 
  •  

Question

I'm trying to pipe several perl programs together, and from everything I've read, piped programs are supposed to all open and run in parallel. That doesn't appear to be the case with whatever it is that I did. Here is the simplified version:

./video

#!/usr/bin/env perl
my $i = 0;
while($i<10){
    my $time = localtime(time);
    print "VID: $time $i\n";
    sleep 1;
    $i++;
}

./controller

#!/usr/bin/env perl
while(my $line = <STDIN>){
    my $time = localtime(time);
    print "CTRL: $time $line";
}

./pipes

#!/usr/bin/env perl
open(my $controller, "|./controller") || die "Can't fork: $!";
open(STDOUT, ">&", $controller);
open(my $video, "|./video") || die "Can't fork: $!";
print STDERR "All processes started\n";

I tried executing it two ways, but both resulted in the same output:

<16:34:21> rswhiting@Minas-Tirith:~/code/Pipes $ ./pipes 
All processes started
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:22 2014 0
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:23 2014 1
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:24 2014 2
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:25 2014 3
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:26 2014 4
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:27 2014 5
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:28 2014 6
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:29 2014 7
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:30 2014 8
CTRL: Sun Apr 20 16:34:32 2014 VID: Sun Apr 20 16:34:31 2014 9
<16:34:49> rob@Minas-Tirith:~/code/Pipes $ ./video | ./controller 
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:15 2014 0
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:16 2014 1
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:17 2014 2
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:18 2014 3
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:19 2014 4
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:20 2014 5
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:21 2014 6
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:22 2014 7
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:23 2014 8
CTRL: Sun Apr 20 16:35:25 2014 VID: Sun Apr 20 16:35:24 2014 9
<16:35:25> rob@Minas-Tirith:~/code/Pipes $

The timestamps from the video program increment as they should, but the time that the control program receives them is identical (after the video program has ended). How do I make the controller receive the data as it is produced instead of after the fact?

Was it helpful?

Solution

It's because of STDOUT buffering in the video program. print is line-buffered only if it goes to an interactive terminal. Otherwise it uses a much bigger buffer, and the accumulated output is emitted in one big chunk when video exits. Try flushing after each print or use autoflush.

$| = 1;  # Enables autoflush

By the way I don't get why you open a pipe TO video, it does not read anything. It should be

system "./video";

If you have a program that you can't modify but does not autoflush, you are in trouble. Except, you're not. You can use a pseudoterminal instead of a pipe. I've never done it. But chapter 64 of the book "The UXIX Programming Interface" is about pseudoterminals.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top