Will Python close an fd if it's out of a local scope?

Question

I found Python close my file descriptor automatically. Run the follow code and use lsof to find the open file. When sleep in function openAndSleep, I found file "fff" was holding by the process. But when it run out of the function, file "fff" was not holding any more.

import time

def openAndSleep():
    f = open("fff", 'w')
    print "opened, sleep 10 sec"
    time.sleep(10)
    print "sleep finish"

openAndSleep()
print "in main...."
time.sleep(10000)

I check class file, it has no __del__ method. It seems strange, anyone know something about it?

Answer 1

Yes, CPython will.

File objects close automatically when their reference count drops to 0. A local scope being cleaned up means that the refcount drops, and if the local scope was the only reference then the file object refcount drops to 0 and is closed.

However, it is better to use the file object as a context manager in a with statement and have it closed automatically that way; don't count on the specific garbage handling implementation of CPython:

def openAndSleep():
    with open("fff", 'w') as f:
        print "opened, sleep 10 sec"
        time.sleep(10)
        print "sleep finish"

Note that __del__ is a hook for custom Python classes; file objects are implemented in C and fill the tp_dealloc slot instead. The file_dealloc() function closes the file object.

If you want to hold a file object open for longer, make sure there is still a reference to it. Store a reference to it somewhere else too. Return it from the function and store the return value, for example, or make it a global, etc.

Answer 2

In short: Yes.

Python spares the user the need to manage his memory by implementing a Garbage Collection mechanism. This basically means that each object in Python will be automatically freed and removed if no one uses it, to free memory and resources so they can be used later in the program again.

File Objects are Pythonic objects, the same as any other object in Python and they too are managed by the garbage collector. Once you leave the function scope the Garbage Collector sees that no one uses the file (using a reference counter) and disposes of the object - which means closing it as well.

What you can do to avoid it is to open the file without using the Python file object by using os.open which returns a file descriptor (int) rather than a Python file object. The file descriptor will then not be discarded by the Garbage Collector since it's not a Python object but an Operating System object and thus your code will work. You should be careful to close (os.close) the fd later, though, or you will leak resources and sooner or later your program will crash (A process can only store 1024 file descriptors and then no more files can be opened)!

Additional information:

http://www.digi.com/wiki/developer/index.php/Python_Garbage_Collection