Encode downstream messages in Netty's IO worker threads

Question

I'm using io.netty-3.9.0.Final with protobuf to create and send messages with large size. I noticed that serialization and writing to CodedOutputStream takes a lot of time, and slows down my application thread.

I expected this kind of work to be processed in Netty's IO worker threads. The Channel.write(Object) documentation says: "Sends a message to this channel asynchronously." Besides, upstream messages decoding occurs in I/O worker thread.

So how can I move encoding from Channel.write() invokation thread to Netty's IO worker threads?

The ChannelPipelineFactory#getPipeline() method looks like this:

public ChannelPipeline getPipeline() throws Exception {
    final ChannelPipeline pipeline = Channels.pipeline();

    // Decoder
    pipeline.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(MAX_FRAME_LENGTH, LENGTH_FIELD_OFFSET, LENGTH_FIELD_LENGTH, LENGTH_ADJUSTMENT, INITIAL_BYTES_TO_STRIP));
    pipeline.addLast("protobufDecoder", new ProtobufDecoder(MessageProto.pMessage.getDefaultInstance()));
    pipeline.addLast("protoMessageDecoder", new ProtoMessageDecoder());

    // Encoder
    pipeline.addLast("frameEncoder", new LengthFieldPrepender(LENGTH_FIELD_LENGTH));
    pipeline.addLast("protobufEncoder", new ProtobufEncoder());
    pipeline.addLast("protoMessageEncoder", new ProtoMessageEncoder());

    pipeline.addLast("handler", upstreamHandler);

    return pipeline;
}

Answer 1

Unfortunately Netty 3.x doesn't work like this. It does send a message asynchronously but the asynchronous part kicks in if Netty is unable to write to the network buffers immediately. In this case it queues the message and will eventually write it out using the I/O thread.

I believe your best option is to upgrade to Netty 4 which I understand has the behaviour you desire. Otherwise you could try adding an OrderedDownstreamThreadPoolExecutor to your pipeline. The write still won't be processed by the I/O thread, but it should ease the pressure on your application thread.