Python Subprocess Finishes, but Output File Not Available

Question

My python code is using a subprocess to call "ifconfig" through the shell and uses ">" to write the output to a text file. When the subprocess finishes, and returns success, I read the output file. I do this at fixed intervals to monitor the network status, but occasionally I'm unable to open the output file. I was just reading that Popen has optional arguments for stdout and stderr, which may be safer/better supported, but I'm curious to why my current version fails. My code is below. There are a few objects and macros from my library included without explanation, but I think the code is still clear enough for this question.

Why does opening the output file occasionally fail? Is it possible the file is not ready when the subprocess returns? What would be a way to guarantee it's ready to open?

       # Build command line expression.
        expr = 'ifconfig' + ' >' + outputFile + ' 2>&1'

        try:
            # Execute command line expression.
            p = subprocess.Popen(expr, shell=True)
        except:
            Error("Unable to open subprocess.")

        if(p is None):
            Error("Unable to create subprocess.")

        # Wait until command line expression has been executed.
        wait = Wait.Wait(Constants.MIN_TIME_TO_QUERY_NETWORK_INFO, Constants.MAX_TIME_TO_QUERY_NETWORK_INFO)

        #Execute command then wait for timeout.    
        if (wait.StopUntilCondition(operator.ne, (p.poll,), None, True)):
            p.kill()
            Error("Get subnet mask subprocess timed out.")

        if(not p.poll() == 0):
            Error("Failed to get network information from operating system.")

        Warning("About to read output file from get subnet mask...")

        # Read temporary output file.
        f = open(outputFile, "r")
        networkInfo = f.read()
        f.close()

Answer 1

To avoid the corrupted/missing output, you should call p.wait() before trying to read the file. You don't need to use a file in this case:

from subprocess import check_output, STDOUT

network_info = check_output('ifconfig', stderr=STDOUT)

If you want to interrupt ifconfig before it is done and read its output; see Stop reading process output in Python without hang?

Answer 2

It is probably easier to just use subprocess.Popen() and write that output to a file yourself. Note that if you want to capture both stdout and stderr you cannot use subprocess.check_output

try:
    p = subprocess.Popen(['ifconfig'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    outdata, errdata = p.communicate()
except Exception as e:
    print "An error occurred:", e
with open(outputFile, 'w') as outf:
    outf.write(outdata)

With this you don't have to wait; communicate won't return before the process is finished.

If you just want to capture stdout, use subprocess.check_output instead;

try:
    outdata = subprocess.check_output(['ifconfig'])
except Exception as e:
    print "An error occurred:", e
with open(outputFile, 'w') as outf:
    outf.write(outdata)