Django, Ajax long polling, Postgresql: idle transaction

Question

I implemented a chat, using ajax long polling and Gevent. To read, the client ajax the update view and wait with Gevent.event.wait for an update.

Problem: The Postgresql transaction opened by Django at the beginning of a request (to get session information) isn't closed until the end of the request. And those idle transactions take a lot of memory.

What would be the cleanest way to close the Postgresql transaction without closing the request ? I'm currently sending the request_finished signal manually but it feels like a hack.

Answer 1

The way you're doing it is probably the the best way within the framework of your hack anyway. Is there any reason you're trying to shoe-horn long-poll into the request-response process instead of using something like django-socketio?

Answer 2

See here: https://docs.djangoproject.com/en/dev/topics/db/transactions/#django.db.transaction.commit_manually

@transaction.commit_manually
def yourview(request):
    # do your db actions
    transaction.commit()

Or if you prefer context managers:

def yourview(request):
    ...
    with transaction.commit_manually():
         # do your db actions
    ...

Also if you're having memory issues holding PostgreSQL connections open you should look a pooling solution using pgbouncer or the various gevent connection pools that exist. You should see some sizable performance gains from doing this.