iPhone How to Set Frame Rate and Slow Down AVCapture didOutputSampleBuffer Delegate

Question

I want to slow down the frame rate of the video device on an iPhone 4S so that the didOutputSampleBuffer delegate is called less frequently. This is to improve performance since I process each frame and need a large frame for the detail.

I tried to use the following to do so when I setup my AVSession:

AVCaptureConnection *conn = [self.output connectionWithMediaType:AVMediaTypeVideo];
[conn setVideoMinFrameDuration:CMTimeMake(1, CAPTURE_FRAMES_PER_SECOND)];
[conn setVideoMaxFrameDuration:CMTimeMake(1, CAPTURE_FRAMES_PER_SECOND)];

But this has no effect, I can change CAPTURE_FRAMES_PER_SECOND from 1 to 60 and see no difference in performance or slowing down of the video capture. Why does this have no effect? How can I slow down the capture frame rate for the video device?

I set up my Session with the following code:

// Define the devices and the session and the settings
self.session = [[AVCaptureSession alloc] init];

//self.session.sessionPreset = AVCaptureSessionPresetPhoto;
//self.session.sessionPreset = AVCaptureSessionPresetHigh;
self.session.sessionPreset = AVCaptureSessionPreset1280x720;

self.device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
self.input = [AVCaptureDeviceInput deviceInputWithDevice:self.device error:nil];

// Add the video frame output
self.output = [[AVCaptureVideoDataOutput alloc] init];
[self.output setAlwaysDiscardsLateVideoFrames:YES];
self.output.videoSettings = [NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA]
                                                        forKey:(id)kCVPixelBufferPixelFormatTypeKey];

// A dispatch queue to get frames
dispatch_queue_t queue;
queue = dispatch_queue_create("frame_queue", NULL);

// Setup the frame rate    
AVCaptureConnection *conn = [self.output connectionWithMediaType:AVMediaTypeVideo];
[conn setVideoMinFrameDuration:CMTimeMake(1, CAPTURE_FRAMES_PER_SECOND)];
[conn setVideoMaxFrameDuration:CMTimeMake(1, CAPTURE_FRAMES_PER_SECOND)];

// Setup input and output and set the delegate to self
[self.output setSampleBufferDelegate:self queue:queue];
[self.session addInput:self.input];
[self.session addOutput:self.output];

// Start the session
[self.session startRunning];

I capture the frames using the "didOutputSampleBuffer" delegate implementation below:

// The delegate method where we get our image data frames from
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
       fromConnection:(AVCaptureConnection *)connection
{

    // Extract a UImage
    CVPixelBufferRef pixel_buffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixel_buffer];

    // Capture the image
    CGImageRef ref = [self.context createCGImage:ciImage fromRect:ciImage.extent];

    // This sets the captured image orientation correctly
    UIImage *image = [UIImage imageWithCGImage:ref scale:1.0 orientation:UIImageOrientationLeft];

    // Release the CGImage
    CGImageRelease(ref);

    // Update the UI on the main thread but throttle the processing
    [self performSelectorOnMainThread:@selector(updateUIWithCapturedImageAndProcessWithImage:) withObject:image waitUntilDone:YES];

}

Answer 1

I am not sure what iOS you are running, but bracket your code like this:

AVCaptureConnection *conn = [self.output connectionWithMediaType:AVMediaTypeVideo];
if ([conn isVideoMaxFrameDurationSupported] && [conn isVideoMinFrameDurationSupported])
{
  [conn setVideoMinFrameDuration:CMTimeMake(1, CAPTURE_FRAMES_PER_SECOND)];
  [conn setVideoMaxFrameDuration:CMTimeMake(1, CAPTURE_FRAMES_PER_SECOND)];
}

else
  NSLog(@"Setting Max and/or Min frame duration is unsupported";

Then go from there. I suspect it is unsupported on your iOS.

Answer 2

This is a partial answer: I believe that there was a change in the Quicktime video capture engine between iOS 5 and iOS 6. In iOS 5, it was possible to capture video at 60 FPS, and there were some apps that made use of this to record for playback in smooth slow motion (e.g. SloPro app). In iOS 6, it was no longer possible to achieve 60 FPS using the same method. There is a lengthy discussion on this issue in the MacRumors Forum thread:

Will jailbreak of iPhone 4S, iOS 6.1, allow recording of 60 FPS video?

Hopefully you can find some information there that might provide a solution to your problem. I'd be very interested to hear if anyone can make this work again. I miss recording at 60 FPS...

Answer 3

It looks like the OP already knew how to set the frame rate but didn't know why the code wasn't working.

The AVCaptureConnection isn't created until you add the input and output to the capture session: AVCaptureConnection documentation.

So I suspect 'conn' is null. Move the code in 'Set the frame rate' after the code in 'Setup input and output and set the delegate to self'.