How to make waitpid block the loop

Question

The following code runs 2 children, who will wait for 10 seconds and terminate. The parent is sitting in a loop, waiting for the children to terminate:

#!/usr/bin/perl

use strict;
use warnings;
use POSIX ":sys_wait_h";

sub func
# {{{
{
      my $i = shift;
      print "$i started\n";
      $| = 1;
      sleep(10);
      print "$i finished\n";
}
# }}}

my $n = 2;
my @children_pids;

for (my $i = 0; $i < $n; $i++) {
      if ((my $pid = fork()) == 0) {
            func($i);
            exit(0);
      } else {
            $children_pids[$i] = $pid;
      }
}

my $stillWaiting;
do {
      $stillWaiting = 0;
      for (my $i = 0; $i < $n; ++$i) {
            if ($children_pids[$i] > 0)
            {
                  if (waitpid($children_pids[$i], WNOHANG) != 0) {
                        # Child is done
                        print "child done\n";
                        $children_pids[$i] = 0;
                  } else {
                        # Still waiting on this child
                        #print "waiting\n";
                        $stillWaiting = 1;
                  }
            }
            #Give up timeslice and prevent hard loop: this may not work on all flavors of Unix
            sleep(0);
      }
} while ($stillWaiting);

print "parent finished\n";

The code is based upon this answer: Multiple fork() Concurrency

It works correctly, but the parent loop is eating processor time. top command gives this:

Here the answer says:

As an additional bonus, the loop will block on waitpid while children are running, so you don't need a busy loop while you wait.

But for me it doesn't block. What's wrong?

Answer 1

You're passing the WNOHANG flag, which makes the call non-blocking. Remove this flag and waitpid will wait at 0% CPU until the child quits.

If you take this approach, you could simplify the code. There's no need to loop until a child is finished, because the blocking waitpid call will do that for you:

for (my $i = 0; $i < $n; ++$i) {
    if ($children_pids[$i] > 0) {
          waitpid($children_pids[$i], 0);
          print "child done\n";
          $children_pids[$i] = 0;
    }
}

Alternatively, change the sleep call to wait for one second. Then your program will check for finished children every second, without pushing up CPU usage.

Answer 2

Since your parent thread isn't actually doing anything while waiting for it's children, I would simplify it to

#!/usr/bin/perl    
use strict;
use warnings;
$| = 1; # autoflush

sub func{
    my($i) = @_;
    print "$i started\n";
    sleep(10);
    print "$i finished\n";
}

my $n = 2;
my @children_pids;

for my $i ( 0..($n-1) ) { # faster, and clearer than the C-style for loop
    my $pid = fork;
    die "Unable to fork" unless defined $pid; # check for errors
    if ( $pid == 0) { # child
        func($i);
        exit(0); # may need to be POSIX::_exit()
    } else { # parent
        push @children_pids, $pid; # don't allow undef or 0 on the list
    }
}

# while( @children_pids ){
#    waitpid shift @children_pids, 0;
# }

waitpid $_, 0 for @children_pids;

print "parent finished\n";

---

If your perl is compiled with IThreads you can use the threads module.
( IThreads is required for fork emulation on Windows )

Use of threads also makes it much easier to do what you originally attempted, joining threads as they finish.

use strict;
use warnings;
use threads (); # not using async
$| = 1; # autoflush

sub func{
    my($i) = @_;
    print "$i started\n";
    sleep(rand(10)); # randomize the order of completion for this example
    print "$i finished\n";
    return $i; # <===
}

my $n = 10;

for my $i ( 0..($n-1) ){
   my $thread = threads->create( \&func, $i ); # ask for scalar value
   die "unable to create thread $i" unless defined $thread;
}

while( threads->list ){
    # join the threads that are done
    for my $thread ( threads->list(threads::joinable) ){
        print 'thread-id: ',  $thread->tid, ' returned: ', $thread->join, "\n";
    }

    # be nice to other threads and processes
    threads->yield;

    # allows the threads to "bunch up" for this example
    # not necessary for real code.
    sleep 2;
}

0 started
1 started
2 started
3 started
4 started
5 started
6 started
7 started
7 finished
8 started
9 started
2 finished
thread-id: 3 returned: 2
thread-id: 8 returned: 7
8 finished
5 finished
thread-id: 6 returned: 5
thread-id: 9 returned: 8
1 finished
thread-id: 2 returned: 1
3 finished
6 finished
9 finished
4 finished
thread-id: 4 returned: 3
thread-id: 5 returned: 4
thread-id: 7 returned: 6
thread-id: 10 returned: 9
0 finished
thread-id: 1 returned: 0