Set AVC/H.264 profile when encoding video in Android using MediaCodec API

Question

Android provides a way to query the supported encoding profiles. But when setting up an encoder I cannot find the way to specify the desired profile to be used.

Finding supported profile/level pairs

Using the MediaCodec API in android you can call getCodecInfo() once you have chosen an encoder component. This returns a MediaCodecInfo object which provides details about the codec component being used. getCapabilitiesForType() returns a CodecCapabilities object, detailing what the codec is capable of. This contains an array of CodecProfileLevels which detail the supported profiles and levels which are supported.

Trying to set the profile

I can't see a field to set the profile for MedieCodec or for the MediaFormat.

There is a KEY_AAC_PROFILE for MediaFormat, but in the reference it explicitly states this is for aac audio only.

In MediaCodec there is a way to pass a Bundle of extra parameters to the codec using setParameters(). This looks like it has no generic documentation, and the parameters which are accepted will be different between codecs (different devices).

Background

Profiles specify a set of encoding features to use. Simple profiles are less computationally intense, but generally sacrifice quality for a given bit rate as a result. The levels specify the maximum resolution / bit-rate which are supported for a given profile. I expected levels usually to be associated with decoding capability, but since it is describing a hardware encoder which has to run in real time, having a maximum setting makes sense to me.

META: (I originally had a lot more links to each class + function I mentioned, but I had to remove them because I don't yet have the rep to post more than 2 links.)

Answer 1

There is no explicit profile selection in the MediaCodec API. Codec will select a profile on its own. It will also select a level based on input width/height and frame-rate numbers, but it may select a level that is higher than minimum required for the configuration.

You can query the encoder codec for the supported levels to see which profiles it may generate, but you will not be able to select a preferred one. Most likely the codec will select the highest profile it can handle for the given level, but there is no guarantee for that.

Answer 2

For Android Kitkat devices we can set desired AVC profile and level into media format as per code snippet below(sets baseline with level 1.3). Please set this before starting MediaCodec.

format.setInteger(MediaFormat.KEY_PROFILE, MediaCodecInfo.CodecProfileLevel.AVCProfileBaseline);
format.setInteger(MediaFormat.KEY_LEVEL, MediaCodecInfo.CodecProfileLevel.AVCLevel13);

Answer 3

Starting with Android 5.0, it is possible to set profile (SDK Level 21) and level (SDK Level 23).

http://developer.android.com/reference/android/media/MediaFormat.html#KEY_PROFILE http://developer.android.com/reference/android/media/MediaFormat.html#KEY_LEVEL

Haven't yet tried how well these work in practice.

Answer 4

Profile and Level hints can be ignored by a given AVC encoder.

To be absolutely sure which profile and level have been selected by the AVC encoder, you can query the SPS in realtime from its MediaFormat after it has started (like during the initial format change event).

See Section 7 of the spec.

This poster shows to pull the profile and level from these base64 strings (please upvote him!):

Can profile-level-id and sprop-parameter-sets be extracted from an RTP stream?

Here are the Base64 encoded SPS and PPS:

...
mIndex = mMediaEncoder.dequeueOutputBuffer(mBufferInfo, TIMEOUT);

// First buffer encoded is the NAL frame metadata
if (mIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) 
{
    Log.i("PPS: "+getParameterSetAsString(mMediaEncoder, "csd-0");
    Log.i("SPS: "+getParameterSetAsString(mMediaEncoder, "csd-1");
}
...

private String getParameterSetAsString(MediaCodec encoder, String csd)
{
    MediaFormat mMediaFormat = encoder.getOutputFormat();
    ByteBuffer ps = mMediaFormat.getByteBuffer(csd);

    // The actual SPS and PPS byte codes
    byte[] mPS = new byte[ps.capacity()-4];
    ps.position(4);
    ps.get(mPS,0,mPS.length);

    //Covert to String
    return Base64.encodeToString(mPS, 0, mPS.length, Base64.NO_WRAP);
}