how to print subprocess' stdout directly to file

Question

How do I force subprocess.Popen/os.Popen to write large output directly to file, without holding anything in the buffer ? I tried this:

os.popen(cmd, bufsize=0)

it didn`t help. Any solution of reading all the output (again, large output) and saving it to file/list would help.

Answer 1

You could use stdout parameter to redirect output from a subprocess to a file:

import shlex
from subprocess import check_call

with open('outputfilename', 'wb', 0) as outputfile:
    check_call(shlex.split(cmd), stdout=outputfile)

Do you know what I should add to the command to prevent the subprocess from printing warning/errors to the shell ?

Just set stderr=subprocess.STDOUT to merge stdout/stderr. Or redirect stderr to os.devnull to discard the output.

Answer 2

This is how to redirect stdout to a file:

import sys
sys.stdout = open('your_file', 'w')

Then everything that would have outputted in the shell is dumped in that file.