Media Codec API with Input as Surface not working with H264 encoder (bigflake example code)

Question

I am trying to run the example code of Media Codec API with H264 Encoder on 4.3 explained in following link of bigflake

http://bigflake.com/mediacodec/CameraToMpegTest.java.txt

I have faced following problem. -> In H264 encoder code the color format,height and width are not getting updated because there is problem in getpatameter implemetation. So i applied this patch (https://code.google.com/p/android/issues/detail?id=58834). -> After applying the patch,also encoder does not encode -> I have seen the observation like D/CameraToMpegTest( 3421): encoder output format changed: {csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], height=144, mime=video/avc, csd-0=java.nio.ByteArrayBuffer[position=0,limit=12,capacity=12], what=1869968451, width=176}

SO why this value is getting changed, No idea... After that we always see encoder gives status of queueOutputBuffer as INFO_TRY_AGAIN_LATER. So it creates the file but it does not encode anything and it stops as

I/MPEG4Writer( 3421): Received total/0-length (0/0) buffers and encoded 0 frames. - video

D/MPEG4Writer( 3421): Stopping Video track

D/MPEG4Writer( 3421): Stopping Video track source

D/MPEG4Writer( 3421): Video track stopped

D/MPEG4Writer( 3421): Stopping writer thread

D/MPEG4Writer( 3421): 0 chunks are written in the last batch

D/MPEG4Writer( 3421): Writer thread stopped

So in my understanding it should work but looks like still encoder is not getting configured properly...

Please guide on this... Thanks

Nehal

Answer 1

I have tried running CameraToMpegTest sample on Android 4.3 emulator. As you'd have realized by now, it's not going to work as it is, and some fixes are required.

1. Implement getparameter properly in SoftAVCEncoder (in case of MIME type - "video/avc") for parameters like width, height, colour format. Otherwise your MediaFormat is not configured properly, and createInputSurface would fail. (I am not sure why this doesn't cause any problem when running H.264 encoding using Mediarecorder)

2. Fix the EGL attributes

3. Most importantly, if you're trying to execute this code in Activity context, make sure you don't block onFrameAvailable callback (final void join() Blocks the current Thread (Thread.currentThread()) until the receiver finishes its execution and dies.)

Answer 2

The "encoder output format changed" message is normal in Android 4.3. That's how the encoder gives you a MediaFormat with csd-0/csd-1 keys, needed by MediaMuxer#addTrack().

Bug 58834 is for the VP8 software encoder; those patches shouldn't be needed for the hardware AVC codec.

The most common reason for INFO_TRY_AGAIN_LATER is lack of input. The encoder may queue up a fair number of input frames before producing any output, so you can't just submit one frame and then wait for output to appear. Turn on the VERBOSE flag and make sure that frames are being submitted.

Answer 3

As the code snippet, you should remove th.join();

    /** Entry point. */
    public static void runTest(CameraToMpegTest obj) throws Throwable {
        CameraToMpegWrapper wrapper = new CameraToMpegWrapper(obj);
        Thread th = new Thread(wrapper, "codec test");
        th.start();
    //    th.join();
        if (wrapper.mThrowable != null) {
            throw wrapper.mThrowable;
        }
    }

It works well for me.