How does Spark use Netty?

Question

I know that Storm now runs on Netty for communication betwen nodes?

Does Apache Spark also use Netty? If so, in what way?

Answer 1

Spark uses Akka Actor for RPC and messaging, which in turn uses Netty.

Also, for moving bulk data, Netty is used.

• For shuffle data, Netty can be optionally used. By default, NIO is directly used to do transfer shuffle data.
• For broadcast data (driver-to-all-worker data transfer), Jetty is used by default.