Capturing images in 24 bpp bitmap from iPhone camera (AVCaptureSession)

Question

I'm capturing frames from the front camera of the iPhone using AVCaptureSession. Im trying to change the format of the AVCaptureVideoDataOutput so it can capture a 24 bpp bitmap. This code provides me a 32 bpp bitmap without any issues:

AVCaptureVideoDataOutput *outputDevice = [[AVCaptureVideoDataOutput alloc] init];
outputDevice.videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey: (id)kCVPixelBufferPixelFormatTypeKey];
[outputDevice setSampleBufferDelegate:self queue:dispatch_get_main_queue()];

However, when I change it to 24, it crashes on that line.

outputDevice.videoSettings = [NSDictionary dictionaryWithObject: [NSNumber numberWithInt:kCVPixelFormatType_24RGB] forKey: (id)kCVPixelBufferPixelFormatTypeKey];

How can I capture the image in 24 bpp? Why does it fail for *kCVPixelFormatType_24RGB*? A workaround would be converting the 32 bmp to 24 but I haven't found out how to do that.

Answer 1

It crashes because kCVPixelFormatType_24RGB isn't supported for iPhone. The only pixel formats that are supported in modern iPhones are:

• kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange
• kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
• kCVPixelFormatType_32BGRA

You can convert any of these to RGB, although the BGRA buffer is simpler to convert. There are various ways to do it (search here and on Google for examples), but here's a pretty straightforward approach:

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
   didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
          fromConnection:(AVCaptureConnection *)connection 
{ 
  @autoreleasepool {
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer,0);   
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width = CVPixelBufferGetWidth(imageBuffer);
    size_t height = CVPixelBufferGetHeight(imageBuffer);
    uint8_t *sourceBuffer = (uint8_t*)CVPixelBufferGetBaseAddress(imageBuffer);
    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
    int bufferSize = bytesPerRow * height;
    uint8_t *bgraData = malloc(bufferSize);
    memcpy(bgraData, sourceBuffer, bufferSize);
    uint8_t *rgbData = malloc(width * height * 3);
    int rgbCount = 0;
    for (int i = 0; i < height; i++) {
      for (int ii = 0; ii < width; ii+=4) {
        int current = (i * height)+ii; 
        rgbData[rgbCount] = bgraData[current + 2]; 
        rgbData[rgbCount + 1] = bgraData[current + 1]; 
        rgbData[rgbCount + 2] = bgraData[current]; 
        rgbCount+=3;
      }
    }
    //
    // Process rgbData
    //
    free (rgbData);
  }
}

Incidentally—it's 8bpp (not 24bpp); three eight-bit planes making up the 24-bit image, or four planes making the 32-bit image. It's also worth pointing out that it's probably easier and quicker, in most cases, just to work with the 32-bit data and ignore the alpha channel, rather than converting to 24-bit.