FFMPEG providing support for HEVC decoding

Question

FFMPEG v2.1 onwards is providing support for HEVC Decoding. I tried an elementary input bin stream as an input for it and yes i got a corresponding YUV file.

Now my question is, since i'm just passing an elementary stream as an input, how is the decoder parsing it?

I have gone through the /libavformat/hevcdec.c which is supposed to be a demuxer for HEVC. I know hevc_probe() is the function in hevcdec.c where it detects if the file can be decoded by the HEVC decoder.

The definition of hevc_probe() function is given below:

static int hevc_probe(AVProbeData *p)
{
    uint32_t code = -1;
    int vps = 0, sps = 0, pps = 0, irap = 0;
    int i;

    for (i = 0; i < p->buf_size - 1; i++) {
        code = (code << 8) + p->buf[i];
        if ((code & 0xffffff00) == 0x100) {
            uint8_t nal2 = p->buf[i + 1];
            int type = (code & 0x7E) >> 1;

            if (code & 0x81) // forbidden and reserved zero bits
                return 0;

            if (nal2 & 0xf8) // reserved zero
                return 0;

            switch (type) {
            case NAL_VPS:        vps++;  break;
            case NAL_SPS:        sps++;  break;
            case NAL_PPS:        pps++;  break;
            case NAL_BLA_N_LP:
            case NAL_BLA_W_LP:
            case NAL_BLA_W_RADL:
            case NAL_CRA_NUT:
            case NAL_IDR_N_LP:
            case NAL_IDR_W_RADL: irap++; break;
            }
        }
    }

    // printf("vps=%d, sps=%d, pps=%d, irap=%d\n", vps, sps, pps, irap);

    if (vps && sps && pps && irap)
        return AVPROBE_SCORE_EXTENSION + 1; // 1 more than .mpg
    return 0;
}

According to whatever i have read, only if this function returns a constant of type AVPROBE_SCORE_EXTENSION, the decoding will proceed. However it is returning AVPROBE_SCORE_EXTENSION+1 Why is it?

Also as seen above in the code, they are deciding some type variable from the input bit code obtained and performing increment in the constants like sps, pps, etc. Is the the normal operation that needs to be performed by a parser which can decode an elementary stream?

It would be really helpful to everyone if anyone is able to give a brief of a parser of a decoder that can decode an elementary stream.

Please Help. Thanks in advance.

Answer 1

Automatically detecting the type of a data stream is a probabilistic matter. A certain probe function looks for files of a certain data type, but can it be absolutely certain that the data conforms to that type? What about false positives? For example, a lot of multimedia files have signatures in the first 4 bytes that consist of 4 ASCII characters. If a probe function checked for those 4 characters and it found a random text file happened to begin with the same 4 characters, that would be a false positive.

Since there are various factors that can influence detection, FFmpeg's probe functions return a certainty value ranging from 0..100 ("definitely does NOT conform" .. "absolute is this type"). AVPROBE_SCORE_EXTENSION is defined as 50. Some file types don't have strong signatures and the best that a probe function can do is check the file's extension.

Where I'm going with all of this: Elementary streams can be a bit tricky to automatically detect. That's why, after gathering enough evidence, the HEVC elementary stream probe detection only has enough confidence to return 51% probability that this stream is HEVC. As FFmpeg runs the stream through various probe functions, another format's probe function could override this one if it returns higher than 51% certainty.

It should be noted that elementary streams are not commonly seen "in the wild". They tend to be packaged into other formats that are better for transport and processing (see program streams and transport streams).