x264 NALUs serialization and handling

Question

I have my x264 encoder, producing NALUs from a raw video stream. I need to send those NALUs over the network. What is the best way of doing so?

Encoder is inserted into a DirectShow graph, it's a transform filter and downstream I have the filter which handles networking. Can I pass NALUs, created by transform filter directly to network "render" filter? Will it create some memory issues?

I would like to know how memory allocated for NALUs is handled inside x264 - who is responsible for freeing it? Also I'm wondering if I can just serialize NALU to a bit stream manually and then rebuild it in the same way?

Answer 1

I need to send those NALUs over the network. What is the best way of doing so?

"Best" needs clarification: easiest to do, best in terms of compatibility, compatible to specific counterpart implementation etc.

Can I pass NALUs, created by transform filter directly to network "render" filter? Will it create some memory issues?

There is no stock network renderer, you should read up on how it needs to be done with specific renderer you are going to use.

I would like to know how memory allocated for NALUs is handled inside x264 - who is responsible for freeing it?

x264 manages buffers it fills, x264_encoder_encode returns you references on those buffers and you don't need to free data, just be sure to copy it out timely since it will be invalidated with next call. Don't forget x264_encoder_close afterwards - it will release all resources managed internally.

Also I'm wondering if I can just serialize NALU to a bit stream manually and then rebuild it in the same way?

Yes you can do it. If your network pair of filters can reproduce the same stream doing network stuff on their inner connection, then it is going to work out fine. The best network protocol in terms of interoperability with H.264 is RTP. It is however pretty complicated if compared to simply accept/send/receive/reproduce steps for a bitstream.

• RTP: A Transport Protocol for Real-Time Applications
• RTP Payload Format for H.264 Video

Answer 2

The best way to send out NALU on to the network would be through an RTP stream. Look at RFC 6184 for details on RTP packetization for H.264. I think you can safely pass NALU to your renderer provided your media buffers are large enough to hold you NALUs.