iPhone App Green Screen Replacement

Question

Q: I'm looking to use the iPhone camera to take a photo and then replace the green screen in that photo with another photo.

What's the best way to dive into this? I couldn't find many resources online.

Thanks in advance!

Answer 1

Conceptually, all that you need to do is loop through the pixel data of the photo taken by the phone, and for each pixel that is not within a certain range of green, copy the pixel into the same location on your background image.

Here is an example I modified from keremic's answer to another stackoverflow question. NOTE: This is untested and just intended to give you an idea of a technique that will work

//Get data into C array
CGImageRef image = [UIImage CGImage];
NSUInteger width = CGImageGetWidth(image);
NSUInteger height = CGImageGetHeight(image);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel_ * width;
NSUInteger bitsPerComponent = 8;
unsigned char *data = malloc(height * width * bytesPerPixel);
// you will need to copy your background image into resulting_image_data. 
// which I am not showing here
unsigned char *resulting_image_data = malloc(height * width * bytesPerPixel);
CGContextRef context = CGBitmapContextCreate(rawData, width, height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);

CGContextDrawImage(context, CGRectMake(0, 0, width, height));
CGContextRelease(context);

//loop through each pixel
for(int row = 0; row < height; row++){
  for(int col = 0; col < width*bytesPerPixel; col=col+4){
    red = data[col];
    green = data[col + 1];
    blue = data[col + 2];
    alpha = data[col + 3];
    // if the pixel is within a shade of green
    if(!(green > 250 && red < 10 && blue < 10)){
      //copy them over to the background image
      resulting_image_data[row*col] = red;
      resulting_image_data[row*col+1] = green;
      resulting_image_data[row*col+2] = blue;
      resulting_image_data[row*col+3] = alpha;
    }
  }
}

//covert resulting_image_data into a UIImage

Answer 2

Have a look at compiling OpenCV for iPhone - not an easy task, but it gives you access to a whole library of really great image processing tools.

I'm using openCV for an app I'm developing at the moment (not all that dissimilar to yours) - for what you're trying to do, openCV would be a great solution, although it requires a bit of learning etc. Once you've got OpenCV working, the actual task of removing green shouldn't be too hard.

Edit: This link will be a helpful resource if you do decide to use OpenCV: Compiling OpenCV for iOS