The problem is that ELBQuery
is a blocking function. If it doesn't yield
another coroutine somewhere, there is no way for the coroutine scheduler to interleave the calls. (That's the whole point of coroutines—they're cooperative, not preemptive.)
If the problem is something like the calc_range
call, that would probably be easy to deal with—break it up into smaller pieces where each one yields to the next, which gives the scheduler a chance to get in between each piece.
But most likely, it's the boto calls that are blocking, and most of your function's time is spent waiting around for get_metric_statistics
to return, while nothing else can run.
So, how do you fix this?
- Spin off a thread for each boto task. Tornado makes it pretty easy to transparently wrap a coroutine around a thread or thread-pool task, which magically unblocks everything. But of course there's a cost to using threads too.
- Schedule the boto tasks on a thread pool instead of a thread apiece. Similar tradeoffs to #1, especially if you only have a handful of tasks. (But if you could be doing 5 tasks each for 500 different users, you probably want a shared pool.)
- Rewrite or monkeypatch boto to use coroutines. This would be the ideal solution… but it's the most work (and the most risk of breaking code you don't understand, and having to maintain it as boto updates, etc.). However, there are people who have at least gotten started on this, like the
asyncboto
project. - Use
greenlets
and monkeypatch enough of the library's dependencies to trick it into being async. This sounds hacky, but it may actually be the best solution; see Marrying Boto to Tornado for this. - Use
greenlets
and monkeypatch the whole stdlib alagevent
to trick boto and tornado to work together without even realizing it. This sounds like a terrible idea; you'd be better off porting your whole app togevent
. - Use a separate process (or even a pool of them) that uses something like
gevent
.
Without knowing more details, I'd suggest looking at #2 and #4 first, but I can't promise they'll turn out to be the best answer for you.