Garbage Collection using Django's Session Middleware

Question

NOTE: Sorry, but I don't have the required reputation score to post more then two links. The snippets posted below reference the following views and supporting class' __del__() method.

I have a function-based view in Django that stores an instance of a class as a value of a django session.

    # First view Function
    if request.method == 'POST':
        logger.info('User %s Began Upload of File %s' % (request.user.username, request.FILES['file']))
        form = uploadFileForm1(request.POST, request.FILES)
        if form.is_valid():
            # form.cleaned_data contains an in memory version of the uploaded file.
            uploaded_shp = ShpUploader(form.cleaned_data['file'])
            # Store ShpUploader instance in cookie to be referenced
            request.session['uploaded_shp'] = uploaded_shp
            return HttpResponseRedirect('./2')
        else:
            for uploadfile_error in form.errors['file']:
                logger.warning(uploadfile_error)

The session is accessed in a later view, where I some methods of this object stored inside the session.

    # Second view Function
    if request.method == 'POST':
        # Required to repass shpPath kwarg
        form = uploadFileForm2(request.POST,shpPath=request.session['uploaded_shp'].upload_full_path)
        if form.is_valid():
            # Pass user-defined field mappings to import_shapefile method.
            request.session['uploaded_shp'].import_shapefile(form.cleaned_data)
            logger.info('Successful - User %s Uploaded File %s' % (request.user.username, request.session['uploaded_shp'].upload_full_path))
            return HttpResponseRedirect('./success')
        else:
            print form.errors

I had initially toyed with the idea of overriding the __del__() method in my class, to automatically delete a folder which this object references.

# Inside my class definition
def __del__(self):
    """
    Given a directory, remove it an its contents.
    Intended to clean up temporary files after upload.
    """
    shutil.rmtree(self.upload_dir)
    logger.info('Delete Successful: %s' % self.upload_dir)

My question is why is my class' __del__() method executed between the first view and the second view, despite storing the actual object inside a session?

I tried to write a basic example using a class with a custom __del__() method, that uses a dictionary to persist an object between functions. This example:

class tester(object):
    def __init__(self, val):
        self.val = val
        print 'created in %s' % self.val

    def __del__(self):
        print 'deleted'

cache = {} 

def f1():
    print 'in f1'
    t = tester('Test Object Value')
    cache['tdict'] = t
    print cache['tdict'].val

def f2():
    print 'in f2'
    print cache['tdict'].val

if __name__ == '__main__': 
    f1()
    f2()

Produces the out put I would have expected, calling the __del__() method only after exiting the second function f2().

in f1
created in Test Object Value
Test Object Value
in f2
Test Object Value
deleted
[Finished in 0.1s]

Am I missing something in regards to how the session middleware works? One thought I had was in regards to weak references, and if the django session middleware uses these? See below for article link to weak references.

# Can't post over two links yet.
http://mindtrove.info/python-weak-references/

Thanks for your insight. I've moved on past this issue, but I'm still curious as to why this behavior is occuring.

Answer 1

__del__ methods are called when there are no more references to an object. And that is exactly the state that your object is in after the end of a request: there are no more references to it in memory. There is a reference in the database row that stores your session information, but Python doesn't care about that, as of course it shouldn't: all it cares about is that as soon as your view function finishes, your object goes out of scope, and is garbage collected.

In your test function, of course, you do have a reference to your object after the end of the function: in your cache dictionary. But that's not the same thing at all as storing in the session.

In other words, this is the wrong way to be going about this. You'll have to implement the clean-up code in the method that builds the shapes. (Storing a file in a session is also a strange thing to do, but let's leave that for now.)