ffmpeg av_seek_frame with AVSEEK_FLAG_ANY causes grey screen

Question

Problem: omxplayer's source code calls the ffmpeg av_seek_frame() method using the AVSEEK_FLAG_BACKWARD flag. Although not 100% sure, I believe this seeks to the closest i-frame. Instead, I want to seek to exact locations, so I modified the source code such that the av_seek_frame() method now uses the AVSEEK_FLAG_ANY flag. Now, when the movie loads, I get a grey screen, generally for 1 second, during which I can hear the audio. I have tried this on multiple computers (I am actually synchronizing them, therefore, at the same time too) so it is not a n isolated incident. My guess is that seeking to non i-frames is computationally more expensive, resulting in the initial grey screen.

Question: How, using ffmpeg, can I instruct the audio to wait until the video is ready before proceeding.

Answer 1

Actually, AVSEEK_FLAG_BACKWARD indicates that you want to find closest keyframe having a smaller timestamp than the one you are seeking.

By using AVSEEK_FLAG_ANY, you get the frame that corresponds exactly to the timestamp you asked for. But this frame might not be a keyframe, which means that it cannot be fully decoded. That explains your "grey screen", that appears until the next keyframe is reached.

The solution would therefore be to seek backward using AVSEEK_FLAG_BACKWARD and, from this keyframe, read the next frames (e.g. using av_read_frame()) until you get to the one corresponding to your timestamp. At this point, your frame would be fully decoded, and would not appear as a "grey screen" anymore.

NOTE: It appears that, for some reason, av_seek_frame() using AVSEEK_FLAG_BACKWARD returns the next keyframe when the frame that I am seeking is the one directly before this keyframe. Otherwise it returns the previous keyframe (which is what I want). My solution is to change the timestamp I give to av_seek_frame() to ensure that it will return the keyframe before the frame I am seeking.

Answer 2

Completing JonesV answer with some code:

void seekFrame(unsigned frameIndex)
{
    // Seek is done on packet dts
    int64_t target_dts_usecs = (int64_t)round(frameIndex
            * (double)m_video_stream->r_frame_rate.den
            / m_video_stream->r_frame_rate.num * AV_TIME_BASE);
    // Remove first dts: when non zero seek should be more accurate
    auto first_dts_usecs = (int64_t)round(m_video_stream->first_dts
        * (double)m_video_stream->time_base.num
        / m_video_stream->time_base.den * AV_TIME_BASE);
    target_dts_usecs += first_dts_usecs;
    int rv = av_seek_frame(
        m_format_ctx, -1, target_dts_usecs, AVSEEK_FLAG_BACKWARD);
    if (rv < 0)
        throw exception("Failed to seek");

    avcodec_flush_buffers(m_codec_ctx);
}

Then you can begin decoding checking AVPacket.dts against original target dts, computed on AVStream.time_base. As soon as you reached the target dts, the next decoded frame should be the desired frame.