Why does Erlang give up on producing a stack trace in the presence of higher order functions?

Question

Erlang produces nice stack traces when something goes wrong, this is helpful when a programmer wants to figure out why it went wrong. In the presence of higher order functions however the mechanism to produce the stack trace seems to fall short. For example compare the two examples below (I added produced stack traces in the comment of the code).

I read about the difficulties of producing stack traces for lazy evaluation (e.g. in Haskell) before. But since Erlang is evaluated strict I had expected a better result here.

My question is: What makes higher order functions a problem for the mechanism Erlang uses to produce a stack trace? And are there other techniques known that would yield a better result?

  1 -module(test).
  2 -export([f/1,a/1]).
  3 
  4 % Auxilary function to print stack trace (by throwing an exception).
  5 
  6 s(X) when X < 0 -> 0.
  7 
  8 % Example 1: Stack trace in presence of a higher order function.
  9 %
 10 % > test:f(1).
 11 % ** exception error: no function clause matching test:s(1) (test.erl, line 6)
 12 %      in function  test:g/2 (test.erl, line 15)
 13 
 14 f(X)   -> h(fun g/2,X).
 15 g(X,Y) -> s(X) - Y.
 16 h(I,X) -> I(X,3).
 17 
 18 % Example 2: Stack trace for chain of 1st order function applications.
 19 % 
 20 % > test:a(1).           
 21 % ** exception error: no function clause matching test:s(1) (test.erl, line 6)
 22 %      in function  test:c/1 (test.erl, line 28)
 23 %      in call from test:b/1 (test.erl, line 27)
 24 %      in call from test:a/1 (test.erl, line 26)
 25 
 26 a(X) -> b(X) + 1.
 27 b(X) -> c(X) + 2.
 28 c(X) -> s(X) + 3.

Answer 1

This is not a result of using higher order functions per se, but of tail call optimisation. If the very last thing a function does before returning is calling another function, then the Erlang VM optimises away the stack frame, since it is not needed anymore. (This is why you can do recursion without overflowing the stack.)

In your example, both f and h do tail calls, and their stack frames are therefore not visible in the stacktrace. On the other hand, a, b and c do not do tail calls, since they have to perform an addition on the result of the function they are calling before returning to the caller.