Python Tornado - making POST return immediately while async function keeps working

Question

so I have a handler below:

class PublishHandler(BaseHandler):

    def post(self):
        message = self.get_argument("message")
        some_function(message)
        self.write("success")

The problem that I'm facing is that some_function() takes some time to execute and I would like the post request to return straight away when called and for some_function() to be executed in another thread/process if possible.

I'm using berkeley db as the database and what I'm trying to do is relatively simple.

I have a database of users each with a filter. If the filter matches the message, the server will send the message to the user. Currently I'm testing with thousands of users and hence upon each publication of a message via a post request it's iterating through thousands of users to find a match. This is my naive implementation of doing things and hence my question. How do I do this better?

Answer 1

You might be able to accomplish this by using your IOLoop's add_callback method like so:

loop.add_callback(lambda: some_function(message))

Tornado will execute the callback in the next IOLoop pass, which may (I'd have to dig into Tornado's guts to know for sure, or alternatively test it) allow the request to complete before that code gets executed.

The drawback is that that long-running code you've written will still take time to execute, and this may end up blocking another request. That's not ideal if you have a lot of these requests coming in at once.

The more foolproof solution is to run it in a separate thread or process. The best way with Python is to use a process, due to the GIL (I'd highly recommend reading up on that if you're not familiar with it). However, on a single-processor machine the threaded implementation will work just as fine, and may be simpler to implement.

If you're going the threaded route, you can build a nice "async executor" module with a mutex, a thread, and a queue. Check out the multiprocessing module if you want to go the route of using a separate process.

Answer 2

I've tried this, and I believe the request does not complete before the callbacks are called.

I think a dirty hack would be to call two levels of add_callback, e.g.:

  def get(self):
    ...
    def _defered():
      ioloop.add_callback(<whatever you want>)
    ioloop.add_callback(_defered)
    ...

But these are hacks at best. I'm looking for a better solution right now, probably will end up with some message queue or simple thread solution.