Way to force file descriptor to close so that pclose() will not block?

Question 1

I see two solutions here:

The neat one: you fork(), pipe() and execve() (or anything in the exec family of course...) "manually", then it is going to be up to you to decide if you want to let your children become zombies or not. (i.e. to wait() for them or not)
The ugly one: if you're sure you only have one of this child process running at any given time, you could use sysctl() to check if there is any process running with this name before you call pclose()... yuk.

I strongly advise the neat way here, or you could just ask whomever responsible to fix that infinite loop in your third party tool haha.

Good luck!

EDIT:

For you first question: I don't know. Doing some researches on how to find processes by name using sysctl() shoud tell you what you need to know, I myself have never pushed it this far.

For your second and third question: popen() is basically a wrapper to fork() + pipe() + dup2() + execl().

fork() duplicates the process, execl() replaces the duplicated process' image with a new one, pipe() handles inter process communication and dup2() is used to redirect the output... And then pclose() will wait() for the duplicated process to die, which is why we're here.

If you want to know more, you should check this answer where I've recently explained how to perform a simple fork with standard IPC. In this case, it's just a bit more complicated as you have to use dup2() to redirect the standard output to your pipe.

You should also take a look at popen()/pclose() source codes, as they are of course open source.

Finally, here's a brief example, I cannot make it clearer than that:

int    pipefd[2];

pipe(pipefd); 
if (fork() == 0) // I'm the child
{
    close(pipefd[0]);    // I'm not going to read from this pipe
    dup2(pipefd[1], 1);  // redirect standard output to the pipe
    close(pipefd[1]);    // it has been duplicated, close it as we don't need it anymore
    execve()/execl()/execsomething()... // execute the program you want
}
else // I'm the parent
{
    close(pipefd[1]);  // I'm not going to write to this pipe
    while (read(pipefd[0], &buf, 1) > 0) // read while EOF
        write(1, &buf, 1);
    close(pipefd[1]);  // cleaning
}

And as always, remember to read the man pages and to check all your return values.

Again, good luck!

Question 2

Another solution is to kill all your children. If you know that the only child processes you have are processes that get started when you do popen(), then it's easy enough. Otherwise you may need some more work or use the fork() + execve() combo, in which case you will know the first child's PID.

Whenever you run a child process, it's PPID (parent process ID) is your own PID. It is easy enough to read the list of currently running processes and gather those that have their PPID = getpid(). Repeat the loop looking for processes that have their PPID equal to one of your children's PID. In the end you build a whole tree of child processes.

Since you child processes may end up creating other child processes, to make it safe, you will want to block those processes by sending a SIGSTOP. That way they will stop creating new children. As far as I know, you can't prevent the SIGSTOP from doing its deed.

The process is therefore:

function kill_all_children()
{
  std::vector<pid_t> me_and_children;

  me_and_children.push_back(getpid());

  bool found_child = false;
  do
  {
    found_child = false;
    std::vector<process> processes(get_processes());
    for(auto p : processes)
    {
      // i.e. if I'm the child of any one of those processes
      if(std::find(me_and_children.begin(),
                   me_and_children.end(),
                   p.ppid()))
      {
         kill(p.pid(), SIGSTOP);
         me_and_children.push_back(p.pid());
         found_child = true;
      }
    }
  }
  while(found_child);

  for(auto c : me_and_children)
  {
    // ignore ourselves
    if(c == getpid())
    {
      continue;
    }
    kill(c, SIGTERM);
    kill(c, SIGCONT);  // make sure it continues now
  }
}

This is probably not the best way to close your pipe, though, since you probably need to let the command time to handle your data. So what you want is execute that code only after a timeout. So your regular code could look something like this:

void send_data(...)
{
  signal(SIGALRM, handle_alarm);
  f = popen("command", "w");
  // do some work...
  alarm(60);  // give it a minute
  pclose(f);
  alarm(0);   // remove alarm
}

void handle_alarm()
{
  kill_all_children();
}

-- about the alarm(60);, the location is up to you, it could also be placed before the popen() if you're afraid that the popen() or the work after it could also fail (i.e. I've had problems where the pipe fills up and I don't even reach the pclose() because then the child process loops forever.)

Note that the alarm() may not be the best idea in the world. You may prefer using a thread with a sleep made of a poll() or select() on an fd which you can wake up as required. That way the thread would call the kill_all_children() function after the sleep, but you can send it a message to wake it up early and let it know that the pclose() happened as expected.

Note: I left the implementation of the get_processes() out of this answer. You can read that from /proc or with the libprocps library. I have such an implementation in my snapwebsites project. It's called process_list. You could just reap off that class.

Question 3

I'm using popen() to invoke a child process which doesn't need any stdin or stdout, it just runs for a short time to do its work, then it stops all by itself. Arguably, invoking this type of child process should rather be done with system() ? Anyway, pclose() is used afterwards to verify that the child process exited cleanly.

Under certain conditions, this child process keeps on running indefinitely. pclose() blocks forever, so then my parent process is also stuck. CPU usage runs to 100%, other executables get starved, and my whole embedded system crumbles. I came here looking for solutions.

Solution 1 by @cmc : decomposing popen() into fork(), pipe(), dup2() and execl(). It might just be a matter of personal taste, but I'm reluctant to rewrite perfectly fine system calls myself. I would just end up introducing new bugs.

Solution 2 by @cmc : verifying that the child process actually exists with sysctl(), to make sure that pclose() will return successfully. I find that this somehow sidesteps the problem from the OP @WilliamKF - there is definitely a child process, it just has become unresponsive. Forgoing the pclose() call won't solve that. [As an aside, in the 7 years since @cmc wrote this answer, sysctl() seems to have become deprecated.]

Solution 3 by @Alexis Wilke : killing the child process. I like this approach best. It basically automates what I did when I stepped in manually to resuscitate my dying embedded system. The problem with my stubborn adherence to popen(), is that I get no PID from the child process. I have been trying in vain with

waitid(P_PGID, getpgrp(), &child_info, WNOHANG);

but all I get on my Debian Linux 4.19 system is EINVAL.

So here's what I cobbled together. I'm searching for the child process by name; I can afford to take a few shortcuts, as I'm sure there will only be one process with this name. Ironically, commandline utility ps is invoked by yet another popen(). This won't win any elegance prizes, but at least my embedded system stays afloat now.

FILE* child = popen("child", "r");
if (child)
{
    int nr_loops;
    int child_pid;
    for (nr_loops=10; nr_loops; nr_loops--)
    {
        FILE* ps = popen("ps | grep child | grep -v grep | grep -v \"sh -c \" | sed \'s/^ *//\' | sed \'s/ .*$//\'", "r");
        child_pid = 0;
        int found = fscanf(ps, "%d", &child_pid);
        pclose(ps);
        if (found != 1)
            // The child process is no longer running, no risk of blocking pclose()
            break;
        syslog(LOG_WARNING, "child running PID %d", child_pid);
        usleep(1000000); // 1 second
    }
    if (!nr_loops)
    {
        // Time to kill this runaway child
        syslog(LOG_ERR, "killing PID %d", child_pid);
        kill(child_pid, SIGTERM);
    }
    pclose(child); // Even after it had to be killed
} /* if (child) */

I learned in the hard way, that I have to pair every popen() with a pclose(), otherwise I pile up the zombie processes. I find it remarkable that this is needed after a direct kill; I figure that's because according to the manpage, popen() actually launches sh -c with the child process in it, and it's this surrounding sh that becomes a zombie.