How do I persist to disk a temporary file using Python?

Question

I am attempting to use the 'tempfile' module for manipulating and creating text files. Once the file is ready I want to save it to disk. I thought it would be as simple as using 'shutil.copy'. However, I get a 'permission denied' IOError:

>>> import tempfile, shutil
>>> f = tempfile.TemporaryFile(mode ='w+t')
>>> f.write('foo')
>>> shutil.copy(f.name, 'bar.txt')

Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    shutil.copy(f.name, 'bar.txt')
  File "C:\Python25\lib\shutil.py", line 80, in copy
    copyfile(src, dst)
  File "C:\Python25\lib\shutil.py", line 46, in copyfile
    fsrc = open(src, 'rb')
IOError: [Errno 13] Permission denied: 'c:\\docume~1\\me\\locals~1\\temp\\tmpvqq3go'
>>> 

Is this not intended when using the 'tempfile' library? Is there a better way to do this? (Maybe I am overlooking something very trivial)

Answer 1

The file you create with TemporaryFile or NamedTemporaryFile is automatically removed when it's closed, which is why you get an error. If you don't want this, you can use mkstemp instead (see the docs for tempfile).

>>> import tempfile, shutil, os
>>> fd, path = tempfile.mkstemp()
>>> os.write(fd, 'foo')
>>> os.close(fd)
>>> shutil.copy(path, 'bar.txt')
>>> os.remove(path)

Answer 2

hop is right, and dF. is incorrect on why the error occurs.

Since you haven't called f.close() yet, the file is not removed.

The doc for NamedTemporaryFile says:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).

And for TemporaryFile:

Under Unix, the directory entry for the file is removed immediately after the file is created. Other platforms do not support this; your code should not rely on a temporary file created using this function having or not having a visible name in the file system.

Therefore, to persist a temporary file (on Windows), you can do the following:

import tempfile, shutil
f = tempfile.NamedTemporaryFile(mode='w+t', delete=False)
f.write('foo')
file_name = f.name
f.close()
shutil.copy(file_name, 'bar.txt')
os.remove(file_name)

The solution Hans Sjunnesson provided is also off, because copyfileobj only copies from file-like object to file-like object, not file name:

shutil.copyfileobj(fsrc, fdst[, length])

Copy the contents of the file-like object fsrc to the file-like object fdst. The integer length, if given, is the buffer size. In particular, a negative length value means to copy the data without looping over the source data in chunks; by default the data is read in chunks to avoid uncontrolled memory consumption. Note that if the current file position of the fsrc object is not 0, only the contents from the current file position to the end of the file will be copied.

Answer 3

Starting from python 2.6 you can also use NamedTemporaryFile with the delete= option set to False. This way the temporary file will be accessible, even after you close it.

Note that on Windows (NT and later) you cannot access the file a second time while it is still open. You have to close it before you can copy it. This is not true on Unix systems.

Answer 4

You could always use shutil.copyfileobj, in your example:

new_file = open('bar.txt', 'rw')
shutil.copyfileobj(f, new_file)