Why Java lmax disruptor have a big latency in passing events through EventProcessors?

Question

I have a ring buffer in my project, in which a lot of publishers will publish events(for example 500 publishers), and I have 3 EventProcessors which should process the events sequentially. All events should pass this way:

{A lot of Publishers} -> {UpStreamProcessor} -> {DownStreamProcessor} -> {logProcessor}

The problem is that I am losing a lot of time in passing events between publish and starting of UpStreamProcessor, and end of UpStreamProcessor to start of DownStreamProcessor.

For example, When I have 500 publishers, It lasts 1ms in average for processing in UpStreamProcessor and DownStreamProcessor, while it lasts 400ms between UpStreamProcessor finish time to DownStreamProcessor start time.

This is the piece of code for constructing ring buffer and processors:

SequenceBarrier sequenceBarrier;

receiveBuffer = new RingBuffer<>(
    MessageContext.FACTORY, 
    new MultiThreadedLowContentionClaimStrategy(inputBufferSize),
    new YieldingWaitStrategy()
);

upStreamAgentProcessor = new BatchEventProcessor<>(
    receiveBuffer,
    receiveBuffer.newBarrier(),
    new UpStreamAgent()
);
sequenceBarrier = receiveBuffer.newBarrier(
    upStreamAgentProcessor.getSequence()
);

downStreamAgentProcessor = new BatchEventProcessor<MessageContext>(
    receiveBuffer,
    sequenceBarrier,
    new DownStreamAgent()
);
sequenceBarrier = receiveBuffer.newBarrier(
    downStreamAgentProcessor.getSequence()
);

logMapAgentProcessor = new BatchEventProcessor<MessageContext>(
    receiveBuffer,
    sequenceBarrier,
    LogMap.getInstance()
);


receiveBuffer.setGatingSequences(logMapAgentProcessor.getSequence());

operationalExecutor.submit(upStreamAgentProcessor);
operationalExecutor.submit(downStreamAgentProcessor);
operationalExecutor.submit(logMapAgentProcessor);

Answer 1

Disruptor is design to handle messages which take 0.0001 ms If a 1 ms or even 0.1 ms delay doesn't bother you I would use a plain ExecutorService. If you are seeing delays or more than 0.001 ms, it is unlikely to be disruptor and the tasks you are performing are taking are too long.

Here is a good presentation on co-ordinated omission. http://www.infoq.com/presentations/latency-pitfalls The bad news is that if you have a bottleneck which slows the producer as you appear to, latency can be much worse than you are measuring.