Java NIO good for low latency or high throughput?

Question

I am new to Java NIO and have used it a little. I have a general query. If you are designing a ultra low latency app vs high throughput application, which of the two gets clearly benefited by using Non blocking IO?

My understanding is that Non blocking IO should certainly help in high throughput as worker threads are not blocking, hence not waiting for response and are free to fire new requests until previous requests are served. Once we get responses for previously fired requests, worker threads can process them asynchronously, increasing throughput.

However, I am unable to see how Non blocking IO directly benefits low latency application.

I guess "Asynchronous behavior is a great way to avoid contention." If that is the case, low contention means low latency. Hence NIO may help in low latency. Does it make sense?

Answer 1

"Asynchronous behavior is a great way to avoid contention." - only when single thread is used. If many threads, contention is unavoidable. You have to use multitrhreading (with or without NIO) to get high throughput and/or low latency.

NIO only helps to keep number of threads low (around the number of available processors) and thus saves memory (each thread consume a lot of memory) and allows enormous number of simultaneous connections, but usually has worse performance than blocking IO.

Answer 2

It all depends on... :)

• If you have a number of simultaneous threads <= number of CPU cores, then low latency you get can using blocking IO/NIO. Usually, throughput is slightly worse in case of blocking IO/NIO. See for instance: http://vanillajava.blogspot.ru/2011/08/comparing-java-7-async-nio-with-nio.html

• If you have a large number of thread, thread-per-connection model becomes less preferred over async NIO. Mostly because you pay the cost of the context switching, which may be not so small (time to call the system scheduler, more cache misses, etc). See, for instance: http://www.cs.rochester.edu/u/cli/research/switch.pdf And more threads = more total price :) Here we need async NIO for both latency and throughput.