Recent findings:
I recently discovered the callback on the client side has different kinds of processing based on what is visible on the user screen. This was blocking the callbacks to be acknowledged. That explains the buffer overflow as well. The frequency of notifications sent from the server is very high compared to the time taken by each client to process them.
Apart from the changes I did like disabling security, configuring timeouts correctly, following line of code helped a lot:
public void OnNotification(AmigoMessage messsage)
{
ThreadPool.QueueUserWorkItem((x) => { ProcessNotification(messsage); });
}