python subprocess poll() is not returning None even if Popen is still running

Question

I have a python script that executes linux commands with timeout using a while loop and sleep like below

fout = tempfile.TemporaryFile()
try:
    p = subprocess.Popen(["/bin/bash","-c", options.command], bufsize=-1, shell=False, preexec_fn=os.setsid, stdin=subprocess.PIPE, stdout=fout, stderr=subprocess.PIPE)
except:
    sys.exit(UNEXPECTED_ERROR)
if options.timeout:
    print "options.timeout = %s" % options.timeout
    elapsed = 0
    time.sleep(0.1) # This sleep is for the delay between Popen and poll() functions
    while p.poll() is None:
        time.sleep(1)
        elapsed = elapsed + 1
        print "elapsed = %s" % elapsed
        if elapsed >= options.timeout:
            # TIMEDOUT
            # kill all processes that are in the same child process group
            # which kills the process tree
            pgid = os.getpgid(p.pid)    
            os.killpg(pgid, signal.SIGKILL)
            p.wait()
            fout.close()
            sys.exit(TIMEOUT_ERROR)
            break
else:
    p.wait()

fout.seek(0) #rewind to the beginning of the file
print fout.read(),
fout.close()
sys.exit(p.returncode)

$ time myScript -c "cat file2" 2>&1 -t 5
options.timeout = 5
elapsed = 1

real    0m11.811s
user    0m0.046s
sys     0m1.153s

My question is in that above case even if the timeout is 5 seconds cat continues till it finishes. Am I missing something here? Please help.

Answer 1

It works as expected on Ubuntu:

$ /usr/bin/ssh root@localhost -t 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ /usr/bin/time python2.4 myscript.py 'cat big_file'
timeout
done
0.01user 0.63system 0:05.16elapsed 12%CPU 

$ /usr/bin/ssh root@localhost -t 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ /usr/bin/time cat big_file >/dev/null
0.02user 0.82system 0:09.93elapsed 8%CPU

It also work with a shell command:

$ /usr/bin/time python2.4 myscript.py 'while : ; do sleep 1; done'
timeout
done
0.02user 0.00system 0:05.03elapsed 0%CPU

Assumptions:

• you can't use time.time() due to possibility of a system clock change

• time.clock() doesn't measure children times on Linux

• we can't emulate time.monotonic() from Python 3.3 in pure Python due to ctypes is not available on Python 2.4

• it is acceptable to survive hibernation e.g., 2 seconds before hibernation + 3 seconds after computer wakes up whenever it happens if timeout is 5 seconds.

#!/usr/bin/env python2.4
import os
import signal
import sys
import tempfile
import time
from subprocess import Popen

class TimeoutExpired(Exception):
    pass

def wait(process, timeout, _sleep_time=.1):
    for _ in xrange(int(timeout * 1. / _sleep_time + .5)):
        time.sleep(_sleep_time)  # NOTE: assume it doesn't wake up earlier
        if process.poll() is not None:
            return process.wait()
    raise TimeoutExpired  # NOTE: timeout precision is not very good

f = tempfile.TemporaryFile() 
p = Popen(["/bin/bash", "-c", sys.argv[1]], stdout=f, preexec_fn=os.setsid,
          close_fds=True)
try:
    wait(p, timeout=5)
except TimeoutExpired:
    print >>sys.stderr, "timeout"
    os.killpg(os.getpgid(p.pid), signal.SIGKILL)
    p.wait()
else:
    f.seek(0)
    for line in f:
        print line,
f.close()  # delete it
print >>sys.stderr, "done"

Answer 2

Beside of the problems I see in your code

• you call Popen() with stdin=subprocess.PIPE and stderr=subprocess.PIPE. But you never handle these pipes. With a command like cat file2, this should be fine, but it can lead to problems.

I can spot a potential misbehaviour: you might have mixed up indentation (as in the 1st version of your question). Assume you have the following:

while p.poll() is None:
    time.sleep(1)
    elapsed = elapsed + 1
    print "elapsed = %s" % elapsed
    if elapsed >= options.timeout:
        # TIMEDOUT
        # kill all processes that are in the same child process group
        # which kills the process tree
        pgid = os.getpgid(p.pid)    
        os.killpg(pgid, signal.SIGKILL)
    p.wait()
    fout.close()
    sys.exit(TIMEOUT_ERROR)
    break

You don't reach the timeout threshold, and nevertheless p.wait() is called due to a bad indentation. Don't mix up tabs and spaces; PEP 8 suggests to use spaces only and a indentation depth of 4 columns.