iOS - Extract pictures from short video give me the same picture

Question

I'm working on an iOS project that capture videos and extract images from this video (for example 10 pictures every 500 ms). My problem is for short videos (few seconds). I'm extracting the same pictures 2 or 3 times. I don't have this problem with video lengths around 10 seconds and more.

When discussing with a friend, he told me it could be a problem with key frame number in video. I'm using AVCaptureDeviceInput, AVCaptureSession and AVCaptureMovieFileOutput to take the video.

So, my questions are : How can I figure out why i'm extracting the same pictures many times ? Is it a key-frame number problem and is it possible to increase this value (from capture session ? from capture device ?).

EDIT : Here is the code for the picture extract :

AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:videoUrl options:nil];
AVAssetImageGenerator *generateImg = [[AVAssetImageGenerator alloc] initWithAsset:asset];

NSMutableArray *pictList = [NSMutableArray array];
for (int i = 0; i < timeList.count; i++) {
    NSError *error = NULL;
    CMTime time = CMTimeMake([[timeList objectAtIndex:i] intValue], 1000);
    CGImageRef refImg = [generateImg copyCGImageAtTime:time actualTime:NULL error:&error];
    //NSLog(@"error==%@, Refimage==%@", error, refImg);

    [pictList addObject:[[UIImage alloc] initWithCGImage:refImg]];
}

And here is the timeList print out :

(lldb) po timeList
<__NSArrayM 0x175afab0>(
0,
131,
262,
393,
524
)

Thanks in advance !

Answer 1

According to the documentation, there is 2 property on the AVAssetImageGenerator. These properties allow to define a tolerance for getting the nearest frame. But if I put this tolerance to zero, it generates exactly the frame I request. Here is the corrected instanciation of the AVAssetImageGenerator :

AVAssetImageGenerator *generateImg = [[AVAssetImageGenerator alloc] initWithAsset:asset];
generateImg.requestedTimeToleranceBefore = kCMTimeZero;
generateImg.requestedTimeToleranceAfter = kCMTimeZero;

It could be slower with big videos, but in my case i don't have time stakes so it's perfect.

Answer 2

I guess you're running into frame rate issues if you're requesting up to 10 frames every 500ms. This could be due to low frame rate source data or it could be due to a calculation issue causing the frames requested to be too close together.

Consider getting the nominalFrameRate of the tracksWithMediaCharacteristic: of the asset and working with multiples of that frame rate.