write to file with tee command in python

Question

I have a python script with variable names e.g V and It. I make a filename based on the parameter as following:

 file_out=io.open("file_Iter" + str(It) + "_V_" +str(V)+".txt", 'w')

Then I'd like to redirect all my terminal output to a this file, so I use this command:

 os.system("echo - START RUN $(LANG=en_US date +%b_%d_%Y_%k_%M)- | tee -a $file_out")

The file_out is created and also the echo command is shown correctly on the terminal but it is not written to file_out. If in tee command, I put e.g tee testfile.txt, then this file is created and also echo command writes in it.

Q: How should I change tee in order to write to the file that is created by variable names?

Answer 1

I'm not sure I understood it correctly, but I guess that what you want to do is the following:

fileName = "file_Iter" + str(It) + "_V_" +str(V)+".txt"
file_out=io.open(fileName, 'w')
os.system("echo - START RUN $(LANG=en_US date +%b_%d_%Y_%k_%M)- | tee -a " + fileName)

Pay attention to the end of the command, where fileName is concatenated.

Answer 2

$file_out refers to a shell variable -- a variable in the context of the shell that you're executing this command. However, file_out is a Python variable in the context of your Python script.

So, we need to pass this from Python into the shell. Luckily, Python provides an easy way to do this with string formatting commands. This will do what you want, I think:

os.system("echo - START RUN $(LANG=en_US date +%b_%d_%Y_%k_%M)- | " + 
          "tee -a {}".format(file_out))

However, there are a couple concerns with this. First, you should be using subprocess.Popen instead of os.system -- it's a newer module that's intended to replace os.system.

Also, you should be aware that, depending on the content of V and It, it may be possible for this shell to run unexpected commands. Imagine that V has the value '.txt ; sudo rm -rf / #'. Now your file name is 'file_IterIt_V_.txt ; sudo rm -rf / #.txt', and your shell command is 'echo - START RUN $(LANG=en_US date +%b_%d_%Y_%k_%M)- | tee -a file_IterIt_V_.txt ; sudo rm -rf / #.txt', which will happily remove many files from your computer. If you're going to do this, you should make sure to escape the filename in your shell command.