To get possibly unlimited subprocess' stdout/stderr output separately as soon as it becomes available, you could use twisted
spawnProcess():
#!/usr/bin/env python
from twisted.internet import protocol
from twisted.internet import reactor
class ProcessProtocol(protocol.ProcessProtocol):
def outReceived(self, data):
print 'got stdout:', data
def errReceived(self, data):
print 'got stderr:', data
def processEnded(self, reason):
reactor.stop()
process = ProcessProtocol()
reactor.spawnProcess(process, 'cmd', ['cmd', 'arg 1', 'arg 2'])
reactor.run()
An alternative is to use threads e.g., teed_call()
or use OS specific code e.g., fcntl
module to make the pipes non-blocking on POSIX systems or use Overlapped I/O with named pipes on Windows.