When does twisted adbapi transaction actually get committed?

Question

I am using adbapi in a cyclone web server. First my handler writes some stuff to a SQL database, then it makes to an HTTP request to another web server. If that HTTP request fails, I want the database transaction to rollback. However, I'm not getting that effect. Looking at the documentation, it says

The function will be called in the thread with a twisted.enterprise.adbapi.Transaction, which basically mimics a DB-API cursor. In all cases a database transaction will be commited after your database usage is finished, unless an exception is raised in which case it will be rolled back.

This isn't quite as precise a statement as I would like. At what point exactly is my "database usage finished"? Is that when the handler's self.finish() method is called? When the method passed into ConnectionPool.runInteraction() is done?

Here is my code

class AccountCreationHandler(BaseRequestHandler):
    @cyclone.web.asynchronous
    def post(self, accessKey, *args, **kwargs):
        try:    
            d = connPool.runInteraction(self.saveStuffToDatabase)
            d.addCallback(self.callWebServer)
            d.addCallback(self.formatResult)
            d.addErrback(self.handleFailure)

        except Exception, e:
            self.handleException(e)


    def saveStuffToDatabase(self, txn):
        txn.execute("INSERT INTO Table1 (f1) VALUES ('v1')")


    def callWebServer(self):
        agent = Agent(reactor)
        hdrs = Headers({ "Content-type": ["application/json"] })
        values = json.dumps({ "someField": 123 })
        body = SimpleProducer(values)
        url = "http://somewebserver.com"
        d = agent.request("POST", url, hdrs, body)
        d.addCallback(self.handleWebResponse)
        return d


    def handleWebResponse(self, response):
        if response.code == 200:
            d = Deferred()
            receiver = SimpleReceiver(d)
            response.deliverBody(receiver)
            d.addCallback(self.saveWebServerResults)
            return d
        else:
            raise Exception("web server failed with http status code %d" % response.code)


    def saveWebServerResults(self, body):
        self.results = body


    def formatResult(self):    
        self.finish(self.results)


class SimpleProducer(object):
    implements(IBodyProducer)

    def __init__(self, body):
        self.body = body
        self.length = len(body)

    def startProducing(self, consumer):
        consumer.write(self.body)
        return succeed(None)

    def pauseProducing(self):
        pass

    def stopProducing(self):
        pass


class SimpleReceiver(Protocol):
    def __init__(self, d):
        self.buf = ''
        self.d = d

    def dataReceived(self, data):
        self.buf += data

    def connectionLost(self, reason):
        if type(reason.value) == ResponseDone:
            self.d.callback(self.buf)
        else:
            self.d.errback(reason)

In the case that the web server throws an error or the connection to it times out or basically if the code gets past the saveStuffToDatabase method, nothing gets rolled back when an error occurs.

I'm guessing that means that the transaction is committed when the method passed into ConnectionPool.runInteraction() has finished without throwing an exception. If that's the case, I guess then I would have to put everything, including the call to the web server synchronously inside saveStuffToDatabase()?

Answer 1

Well, I reimplemented the code using synchronous calls and it does appear to work correctly. Looking at the documentation for the runInteraction() method, it's made a little more clear:

def runInteraction(self, interaction, *args, **kw):

Interact with the database and return the result. The 'interaction' is a callable object which will be executed in a thread using a pooled connection. It will be passed an Transaction object as an argument (whose interface is identical to that of the database cursor for your DB-API module of choice), and its results will be returned as a Deferred. If running the method raises an exception, the transaction will be rolled back. If the method returns a value, the transaction will be committed.