Store all stdout to file while still displaying it on screen

Question

There seems to be a question/answer for all languages here except for python (at least I haven't found it)

From this question Redirect stdout to a file in Python? I learned that in order to redirect all output from the screen to a file I just have to use:

import sys

some_list = ['elem1', 'elem2']

for elem in some_list:

    sys.stdout = open(elem + '.log', 'w')

    # Do lots of stuff that print messages.
    print 'lots of stuff for', elem

print 'Code finished'

This stores all output to the elemx.log files:

elem1.log
lots of stuff for elem1

elem2.log
lots of stuff for elem2
Code finished

I have two questions regarding this approach:

1- How can I make the last line printed line (ie: Code finished) not be stored in the file created for the last elem?

2- How can I have the stored output also show on screen?

Answer 1

Wrappers is sexy

import sys

class Logger(file):
    def __init__(self,*a,**kw):
        # copy original stdout to instance
        self.stdout = sys.stdout
        return super(Logger,self).__init__(*a,**kw)

    def write(self,data):
        self.stdout.write(data) # to screen
        return super(Logger,self).write(data) #to file

    def writelines(self,lines):
        for line in lines: self.write(line)

    def close(self):
        # return it back
        sys.stdout = self.stdout



some_list = ['elem1', 'elem2']

for elem in some_list:
    with Logger("/tmp/1/{}.log".format(elem), "w") as sys.stdout:
    # Do lots of stuff that print messages.
        print 'lots of stuff for', elem



print 'Code finished'

Result

$ python2 out.py 
Code finished
$ ls
elem1.log  elem2.log  out.py

Cool side effect:

print 'this on screen'

with Logger("/tmp/1/main.log", "w") as sys.stdout:
     print 'this on screen an in main.log'

     with Logger("/tmp/1/sub.log", "w") as sys.stdout:
          print 'this on screen, in man.log and in sub.log'

print 'only on screen again'

Answer 2

So you can emulate the unix tee functionality with a custom redirect class:

import sys
import os

class OutputRedirect(object):
    def __init__(self):
        self.output_devices = []

    def add_device(self,name):
        self.output_devices.append(open(name,"w+"))

    def remove_device(self,name):
        idx = [f.name for f in self.output_devices].index(name)
        self.output_devices.pop(idx)

    def write(self,msg):
        for device in self.output_devices:
            device.write(msg)

    def __del__(self):
        for device in self.output_devices:
            device.close()


def main():
    outputs = OutputRedirect()
    outputs.add_device("test.txt")
    outputs.add_device("/dev/stdout")

    sys.stdout = outputs
    print "this is a test"
    print "this is another test"

    outputs.remove_device("test.txt")
    print "this is not written to file"

if __name__ == "__main__":
    main()

This will allow you to intercept print statements and do what you like with them. You could even add custom tags to your messages to specify which output devices they should go to. However, as I said in the comments above, logging is likely a better option.

Answer 3

For outputting last time on standart output, just assign sys.stdout to standard output file stream.

import sys

some_list = ['elem1', 'elem2']

for elem in some_list:

    sys.stdout = open(elem + '.log', 'w')

    # Do lots of stuff that print messages.
    print 'lots of stuff for', elem

sys.stdout = open("/dev/stdout", "w")

print 'Code finished'

And for the second problem I don't think there is any other solution, than printing the lines twice, one on standard output and other on file.

import sys

some_list = ['elem1', 'elem2']

for elem in some_list:

    f = open(elem + '.log', 'w')

    # Do lots of stuff that print messages.
    print 'lots of stuff for '+ elem
    f.write( 'lots of stuff for '+ elem)


print 'Code finished'
f.write( 'Code finished')

Answer 4

This produces the desired outcome.

import sys

some_list = ['elem1', 'elem2']

for elem in some_list:
    with open(elem + '.log', 'w') as f:
        # Do lots of stuff that print messages.
        f.write('lots of stuff for ' + elem)
        print 'lots of stuff for', elem

print 'Code finished'

stdout

lots of stuff for elem1
lots of stuff for elem2
Code finished

elem1.log

lots of stuff for elem1

elem2.log

lots of stuff for elem2