ZODB with Tornado

Question

I've got a small web-app built with Tornado where I'd like to use ZODB for some data storage. According to the ZODB docs, multi-threaded programs are supported, but they should start up a new connection per thread. I think that means that I have to do something like

### On startup
dbFilename = os.path.join(os.path.dirname(os.path.abspath(__file__)), "Data.fs")
db = DB(FileStorage(dbFilename))

### Example handler
class Example(tornado.web.RequestHandler):
    def get(self):
        try:
            conn = db.open()
            root = conn.root()
            ### do stuff with root here
            root._p_changed = 1  ## Include these lines for writes
            transaction.commit() ## on sub-elements
        finally:
            conn.close()

Firstly, is the new connection still necessary for all db-interacting handlers or just ones that do writes? Would it be reasonable to start one connection on startup and use it for all my reads, then do the above connection song-and-dance only when I need to write something?

Secondly, what's the idiomatic way of abstracting that pattern in Python? I've got something like

def withDB(fn):
    try:
        conn = db.open()
        root = conn.root()
        res = fn(root)
        root._p_changed = 1
        transaction.commit()
        return res
    finally:
        conn.close()

def delete(formName):
    def local(root):
        ### do stuff with root here
    return withDB(local)

in mind, but that's probably my Lisp showing.

A general head-check on the approach would also be welcome.

Answer 1

You need to create a new connection per thread. The ZODB provides each connection with a consistent per-transaction view (MVCC, multi-view concurrency control), so even for reading you want a separate connection. The connection can be re-used for sequential requests in one thread.

So for the connection, I'd use the per-thread pool supplied by ZODB.DB, perhaps caching the connection per request (as pyramid_zodbconn does).

Within your request handlers, you can use the transaction manager as a context manager:

class Example(tornado.web.RequestHandler):
    def get(self):
        connection = some_connection_pool.get_connection()
        with transaction.manager:
            root = conn.root()
            res = fn(root)
            root._p_changed = 1

Using the transaction.manager object as a context manager ensures that the transaction is started on enter, and committed on exit without exception, aborted on exit with an exception.

You could create a context manager to handle the ZODB connection as well:

from contextlib import contextmanager

@contextmanager
def zodbconn(db):
    conn = db.open()
    yield conn.root()
    conn.close()

then use that as a context manager together with the transaction manager:

class Example(tornado.web.RequestHandler):
    def get(self):
        with zodbconn(db) as root, transaction.manager:
            res = fn(root)
            root._p_changed = 1

This context manager takes the database object, and returns the root object, closing the connection automatically when the context is exited again.