How to overlay GLKView on top of UIImagePickerController

Question

I'm trying to overlay OpenGL ES content, using a GLKView, on top of a live video feed from the camera using a UIImagePickerController. I've read several tutorials, books, and posts but, still can't find the answer. For your reference, some of the posts I've read are here, here and here.

In viewDidLoad I'm doing the following:

// Create an OpenGL ES 2.0 context and provide it to the view
// NOTE: viewOverlay is of class GLKView and a property of the view controller
viewOverlay.context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
viewOverlay.opaque = NO;
viewOverlay.backgroundColor=[UIColor clearColor];

// Make the new context current
[EAGLContext setCurrentContext:viewOverlay.context];

// Create a base effect that provides standard OpenGL ES 2.0
// Shading Language programs and set constants to be used for 
// all subsequent rendering
self.baseEffect = [[GLKBaseEffect alloc] init];
self.baseEffect.useConstantColor = GL_TRUE;
self.baseEffect.constantColor = GLKVector4Make(
  1.0f, // Red
  0.0f, // Green
  0.0f, // Blue
  1.0f);// Alpha

// Set the background color stored in the current context 
glClearColor(0.0f, 0.0f, 0.0f, 0.0f); // background color

And in viewDidAppear I'm doing this:

    UIImagePickerController *uip;
    uip = [[UIImagePickerController alloc] init];
    uip.sourceType = UIImagePickerControllerSourceTypeCamera;
    uip.showsCameraControls = NO;
    uip.toolbarHidden = YES;
    uip.navigationBarHidden = YES;
    uip.wantsFullScreenLayout = YES;
    uip.cameraViewTransform = CGAffineTransformScale(uip.cameraViewTransform, CAMERA_TRANSFORM, CAMERA_TRANSFORM);
   [viewOverlay addSubview:uip.view];
   [viewOverlay sendSubviewToBack:uip.view];
   [viewOverlay bringSubviewToFront:viewOverlay];

If I step through the program, I can see the OpenGL objects get rendered once. But, when UIImagePickerController is added as a sub-view, it's the only thing on screen. If I comment out the last three lines, the OpenGL objects are rendered but, of course the camera isn't visible.

I'd like the camera's video image to be rendered in back of the OpenGL objects I'm drawing. Creating an augmented reality effect. Any and all help would be greatly appreciated!

Mike

Answer 1

What do you want your camera to do - just display a camera view for an AR effect, or actually use/manipulate the image data too? If you need anything other than a simple AR display, use AVFoundation.

I've had loads of trouble with this before and read the same posts you have, so I offer two solutions:

1) Use a Container View

In your storyboard file, create a UIViewController with a UIView for the camera and a Container View on top with a segue connected to a GLKViewController. Link your main UIViewController to your camera code and link your GLKViewController to your OpenGL ES code. Both the UIView and the GLKView inside the container will load and run seperately when the UIViewController is visible.

2) App delegate hack

Setup your UIImagePickerController inside the app delegate, specifically the method:

(BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions

Then, add the subview to your window property. This will always make sure that your camera gets loaded onto your window before any views, and ensures it is at the bottom of the layer stack.