Can Deferred.callback() or Deferred.errback() raise an exception to the caller?

Question

Can there be a condition where calling .callback() or .errback() will raise an exception to the caller where it won't be captured by the deferred?

Say I have the the following deferred and callbacks:

from twisted.internet import defer

def bad_callback(result):
    raise Exception()

def bad_errback(result):
    raise Exception()

d = defer.Deferred()
d.addCallbacks(bad_callback, bad_errback)

If I call d.callback(None), the result in d will be the Exception from bad_callback(). If I call d.errback(Exception()), the result in d will be the Exception raised from bad_errback(). But, in either of those cases the exceptions will not be raised to the caller.

Now, I do know of a couple conditions where calling .callback() or .errback() will raise an exception to the caller, but those are conditions where you violate the proper use of deferreds.

• Obviously, if you call .callback() or .errback() with an improper number of arguments, it will raise a TypeError.

• Calling an already called deferred will raise AlreadyCalledError.

• Calling .callback(defer.Deferred()) will raise an AssertionError.

• Calling .errback() is equivalent to calling .errback(failure.Failure()) which will raise NoCurrentExceptionError if there is no active exception.

Really my question comes down to: can I safely rely on the behavior that calling .callback(result) or .errback(exception_or_failure) with a result will never raise an exception so long as the deferred has not already been called and the result is proper?

Answer 1

I ran your example, adding two lines to the bottom:

d.callback(None)
print("OK!")

and got this output:

Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
  File "callbacks.py", line 11, in <module>
    d.callback(None)
  File ".../twisted/internet/defer.py", line 368, in callback
    self._startRunCallbacks(result)
  File ".../twisted/internet/defer.py", line 464, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File ".../twisted/internet/defer.py", line 551, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "callbacks.py", line 4, in bad_callback
    raise Exception()
exceptions.Exception: 
OK!

So in this specific case (as you have determined yourself), no, the exception won't be re-raised.

In the general case, there are a few places where exceptions will effectively propagate out; if you have a MemoryError because you are totally out of memory, it's likely that the Deferred implementation itself will allocate a little memory somewhere by attempting to call a function or something and that exception will come back to you.

But this is just a risk of programming in Python in general; there are several exceptions (MemoryError, KeyboardInterrupt) which may arise with no warning. If your whole process isn't burning down, then no, callback and errback won't raise exceptions except in the cases you've outlined.