NDB tasklets not visible in appstats

Question

Why is it that the line numbers from methods decorated with @ndb.tasklet are not present in appstats?

In our app we have a convention to include both a synchronous and an asynchronous version of functions, something like:

def do_something(self, param=None):
    return self.do_something_async(param=param).get_result()

@ndb.tasklet
def do_something_async(self, param=None):
    stuff = yield self.do_something_else_async(stuff=param)
    # ...
    raise ndb.Return(stuff)

…but even after setting appegnine_config.appstats_MAX_STACK to something huge, and emptying appengine_config.appstats_RE_STACK_SKIP, still the reports in appstats will leave my application code the first time some_tasklet.get_result() is called.

Here's an example from appstats:

The expanded stack frame at learn.get_list_of_cards_to_learn() simply returns self.get_list_of_cards_to_learn_async().get_result(), which is a tasklet that in turn calls a bunch of other tasklets. However none of those tasklets are visible in appstats, all I see is ndb internals.

I'm not sure how exactly ndb is executing those decorators, but if I put a pdb trace in one of them and run my test suite, I can see the stack frames all the way down to the pdb line I put in the tasklet, so I don't understand why is that not there in appstats.

Some of the requests cause a large amount of RPC calls, but I'm not sure how to figure out which part of my app is making them, as I cannot trace it past the first tasklet in appstats.

Is there something maybe I need to fine-tune in appengine_config?

Answer 1

This has to do with the way tasklets are managed by NDB's scheduler. There's not much you can do about it.