Gunicorn + Subprocesses raises exception [Errno 10]

Question

I've stumbled across a weird exception I haven't been able to resolve... can anyone suggest what is wrong or a new design? I'm running a Gunicorn/Flask application. In the configuration file, I specify some work to do with an on_starting hook [1]. Inside that hook code, I have some code like this (nothing fancy):

# Called before the server is started
my_thread = package.MyThread()
my_thread.start()

The package.MyThread class looks like the following. The ls command is unimportant, it can be any command.

class MyThread(threading.Thread):
  """
    Run a command every 60 seconds.

  """
  def __init__(self):
    threading.Thread.__init__(self)
    self.event = threading.Event()

  def run(self):
    while not self.event.is_set():
      ptest = subprocess.Popen(["ls"], stdout=subprocess.PIPE)
      ptest.communicate()
      self.event.wait(60)

  def stop(self):
    self.event.set()

Upon starting up the server, I'm always presented with this exception:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "__init__.py", line 195, in run
    ptest.communicate()
  File "/usr/lib64/python2.6/subprocess.py", line 721, in communicate
    self.wait()
  File "/usr/lib64/python2.6/subprocess.py", line 1288, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib64/python2.6/subprocess.py", line 462, in _eintr_retry_call
    return func(*args)
OSError: [Errno 10] No child processes

Can anyone suggest what is going on here? I haven't tried implementing the changes in [2], they seemed hacky.

[1] - http://gunicorn.org/configure.html#server-hooks

[2] - Popen.communicate() throws OSError: "[Errno 10] No child processes"

Answer 1

The error turns out to be related to signal handling of SIGCHLD.

The gunicorn arbiter intercepts SIGCHLD, which breaks subprocess.Popen. The subprocess.Popen module requires that SIGCHLD not be intercepted (at least, this is true for Python 2.6 and earlier).

According to bugs.python.org this bug has been fixed in Python 2.7.