why lager design this way?

Question

I meet a problem when I use lager. In the lager source code, the lager_backend_throttle.erl file.

handle_event({log, _Message},State) ->
    {message_queue_len, Len} = erlang:process_info(self(), message_queue_len),
    case {Len > State#state.hwm, State#state.async} of
        {true, true} ->
            %% need to flip to sync mode
            lager_config:set(async, false),
            {ok, State#state{async=false}};
        {false, false} ->
            %% need to flip to async mode
            lager_config:set(async, true),
            {ok, State#state{async=true}};
        _ ->
            %% nothing needs to change
            {ok, State}
    end;

when the message_queue_len is more than Threshold, It will flip to sync mode.

when the message_queue_len is less than Thershold, It will flip to async mode.

I think when there is too many message, It should change the mode to async to process the message more quickly. Why lager design this way?

The reason I guess is that there is limit length in the message_queue, If there is too many message, the process may crash. So lager throttle down the speed of sending message by change the sending mode?

Answer 1

I found the answer on github.

Prior to lager 2.0, the gen_event at the core of lager operated purely in synchronous mode. Asynchronous mode is faster, but has no protection against message queue overload. In lager 2.0, the gen_event takes a hybrid approach. it polls its own mailbox size and toggles the messaging between synchronous and asynchronous depending on mailbox size.

{async_threshold, 20}, {async_threshold_window, 5} This will use async messaging until the mailbox exceeds 20 messages, at which point synchronous messaging will be used, and switch back to asynchronous, when size reduces to 20 - 5 = 15.

If you wish to disable this behaviour, simply set it to 'undefined'. It defaults to a low number to prevent the mailbox growing rapidly beyond the limit and causing problems. In general, lager should process messages as fast as they come in, so getting 20 behind should be relatively exceptional anyway.

If you want to limit the number of messages per second allowed from error_logger, which is a good idea if you want to weather a flood of messages when lots of related processes crash, you can set a limit:

{error_logger_hwm, 50} It is probably best to keep this number small.