Rotate, Change Colors and Get RGB565 data from NSImage

Question 1

Ok, I decided to spend the day researching Peter's suggestion of using CoreImage. I had done some research previously and decided it was too hard but after an entire day of research I finally worked out what I needed to do and amazingly it couldn't be easier.

Early on I had decided that the Apple ChromaKey Core Image example would be a great starting point but the example code frightened me off due to the 3-dimensional colour cube. After watching the WWDC 2012 video on Core Image and finding some sample code on github (https://github.com/vhbit/ColorCubeSample) I decided to jump in and just give it a go.

Here are the important parts of the working code, I haven't included the RGB565Data method as I haven't written it yet, but it should be easy using the method Peter suggested:

CIImage+Extras.h

- (NSImage*) NSImage;
- (CIImage*) imageRotated90DegreesClockwise:(BOOL)clockwise;
- (CIImage*) imageWithChromaColor:(NSColor*)chromaColor BackgroundColor:(NSColor*)backColor;
- (NSColor*) colorAtX:(NSUInteger)x y:(NSUInteger)y;

CIImage+Extras.m

- (NSImage*) NSImage
{
    CGContextRef cg = [[NSGraphicsContext currentContext] graphicsPort];
    CIContext *context = [CIContext contextWithCGContext:cg options:nil];
    CGImageRef cgImage = [context createCGImage:self fromRect:self.extent];

    NSImage *image = [[NSImage alloc] initWithCGImage:cgImage size:NSZeroSize];

    return [image autorelease];
}

- (CIImage*) imageRotated90DegreesClockwise:(BOOL)clockwise
{
    CIImage *im = self;
    CIFilter *f = [CIFilter filterWithName:@"CIAffineTransform"];
    NSAffineTransform *t = [NSAffineTransform transform];
    [t rotateByDegrees:clockwise ? -90 : 90];
    [f setValue:t forKey:@"inputTransform"];
    [f setValue:im forKey:@"inputImage"];
    im = [f valueForKey:@"outputImage"];

    CGRect extent = [im extent];
    f = [CIFilter filterWithName:@"CIAffineTransform"];
    t = [NSAffineTransform transform];
    [t translateXBy:-extent.origin.x
                yBy:-extent.origin.y];
    [f setValue:t forKey:@"inputTransform"];
    [f setValue:im forKey:@"inputImage"];
    im = [f valueForKey:@"outputImage"];

    return im;
}

- (CIImage*) imageWithChromaColor:(NSColor*)chromaColor BackgroundColor:(NSColor*)backColor
{
    CIImage *im = self;

    CIColor *backCIColor = [[CIColor alloc] initWithColor:backColor];
    CIImage *backImage = [CIImage imageWithColor:backCIColor];
    backImage = [backImage imageByCroppingToRect:self.extent];
    [backCIColor release];
    float chroma[3];

    chroma[0] = chromaColor.redComponent;
    chroma[1] = chromaColor.greenComponent;
    chroma[2] = chromaColor.blueComponent;

    // Allocate memory
    const unsigned int size = 64;
    const unsigned int cubeDataSize = size * size * size * sizeof (float) * 4;
    float *cubeData = (float *)malloc (cubeDataSize);
    float rgb[3];//, *c = cubeData;

    // Populate cube with a simple gradient going from 0 to 1
    size_t offset = 0;
    for (int z = 0; z < size; z++){
        rgb[2] = ((double)z)/(size-1); // Blue value
        for (int y = 0; y < size; y++){
            rgb[1] = ((double)y)/(size-1); // Green value
            for (int x = 0; x < size; x ++){
                rgb[0] = ((double)x)/(size-1); // Red value
                float alpha = ((rgb[0] == chroma[0]) && (rgb[1] == chroma[1]) && (rgb[2] == chroma[2])) ? 0.0 : 1.0;

                cubeData[offset]   = rgb[0] * alpha;
                cubeData[offset+1] = rgb[1] * alpha;
                cubeData[offset+2] = rgb[2] * alpha;
                cubeData[offset+3] = alpha;

                offset += 4;
            }
        }
    }

    // Create memory with the cube data
    NSData *data = [NSData dataWithBytesNoCopy:cubeData
                                        length:cubeDataSize
                                  freeWhenDone:YES];
    CIFilter *colorCube = [CIFilter filterWithName:@"CIColorCube"];
    [colorCube setValue:[NSNumber numberWithInt:size] forKey:@"inputCubeDimension"];
    // Set data for cube
    [colorCube setValue:data forKey:@"inputCubeData"];

    [colorCube setValue:im forKey:@"inputImage"];
    im = [colorCube valueForKey:@"outputImage"];

    CIFilter *sourceOver = [CIFilter filterWithName:@"CISourceOverCompositing"];
    [sourceOver setValue:im forKey:@"inputImage"];
    [sourceOver setValue:backImage forKey:@"inputBackgroundImage"];

    im = [sourceOver valueForKey:@"outputImage"];

    return im;
}

- (NSColor*)colorAtX:(NSUInteger)x y:(NSUInteger)y
{
    NSBitmapImageRep* bitmap = [[NSBitmapImageRep alloc] initWithCIImage:self];
    NSColor *color = [bitmap colorAtX:x y:y];
    [bitmap release];
    return color;
}

Question 2

For most of this, you'll want to use Core Image.

Rotation you can do with the CIAffineTransform filter. This takes an NSAffineTransform object. You may have already worked with that class before. (You could do the rotation with NSImage itself, but it's easier with Core Image and you'll probably need to use it for the next step anyway.)

I don't know what you mean by “change the colour of pixels that are one colour to another colour”; that could mean any of a lot of different things. Chances are, though, there's a filter for that.

I also don't know why you need 565 data specifically, but assuming you have a real need for that, you're correct that that function will be involved. Use CIContext's lowest-level rendering method to get 8-bit-per-component ARGB output, and then use that vImage function to convert it to 565 RGB.

Question 3

I have managed to get what I want by using NSBitmapImageRep (accessing it with a bit of a hack). If anyone knows a better way of doing this, please do share.

The - (NSBitmapImageRep)bitmap method is my hack. The NSImage starts of having only an NSBitmapImageRep, however after the rotation method a CIImageRep is added which takes priority over the NSBitmapImageRep which breaks the colour code (as NSImage renders the CIImageRep which doesn't get colored).

BitmapImage.m (Subclass of NSImage)

CGContextRef CreateARGBBitmapContext (CGImageRef inImage)
{
    CGContextRef    context = NULL;
    CGColorSpaceRef colorSpace;
    void *          bitmapData;
    int             bitmapByteCount;
    int             bitmapBytesPerRow;


    size_t pixelsWide = CGImageGetWidth(inImage);
    size_t pixelsHigh = CGImageGetHeight(inImage);
    bitmapBytesPerRow   = (int)(pixelsWide * 4);
    bitmapByteCount     = (int)(bitmapBytesPerRow * pixelsHigh);

    colorSpace = CGColorSpaceCreateDeviceRGB();
    if (colorSpace == NULL)
        return nil;

    bitmapData = malloc( bitmapByteCount );
    if (bitmapData == NULL)
    {
        CGColorSpaceRelease( colorSpace );
        return nil;
    }
    context = CGBitmapContextCreate (bitmapData,
                                     pixelsWide,
                                     pixelsHigh,
                                     8,
                                     bitmapBytesPerRow,
                                     colorSpace,
                                     kCGImageAlphaPremultipliedFirst);
    if (context == NULL)
    {
        free (bitmapData);
        fprintf (stderr, "Context not created!");
    }
    CGColorSpaceRelease( colorSpace );

    return context;
}

- (NSData *)RGB565Data
{
    CGContextRef cgctx = CreateARGBBitmapContext(self.CGImage);
    if (cgctx == NULL)
        return nil;

    size_t w = CGImageGetWidth(self.CGImage);
    size_t h = CGImageGetHeight(self.CGImage);
    CGRect rect = {{0,0},{w,h}};
    CGContextDrawImage(cgctx, rect, self.CGImage);

    void *data = CGBitmapContextGetData (cgctx);
    CGContextRelease(cgctx);

    if (!data)
        return nil;

    vImage_Buffer src;
    src.data = data;
    src.width = w;
    src.height = h;
    src.rowBytes = (w * 4);

    void* destData = malloc((w * 2) * h);

    vImage_Buffer dst;
    dst.data = destData;
    dst.width = w;
    dst.height = h;
    dst.rowBytes = (w * 2);

    vImageConvert_ARGB8888toRGB565(&src, &dst, 0);

    size_t dataSize = 2 * w * h; // RGB565 = 2 5-bit components and 1 6-bit (16 bits/2 bytes)
    NSData *RGB565Data = [NSData dataWithBytes:dst.data length:dataSize];

    free(destData);

    return RGB565Data;
}

- (NSBitmapImageRep*)bitmap
{
    NSBitmapImageRep *bitmap = nil;

    NSMutableArray *repsToRemove = [NSMutableArray array];

    // Iterate through the representations that back the NSImage
    for (NSImageRep *rep in self.representations)
    {
        // If the representation is a bitmap
        if ([rep isKindOfClass:[NSBitmapImageRep class]])
        {
            bitmap = [(NSBitmapImageRep*)rep retain];
            break;
        }
        else
        {
            [repsToRemove addObject:rep];
        }
    }

    // If no bitmap representation was found, we create one (this shouldn't occur)
    if (bitmap == nil)
    {
        bitmap = [[[NSBitmapImageRep alloc] initWithCGImage:self.CGImage] retain];
        [self addRepresentation:bitmap];
    }

    for (NSImageRep *rep2 in repsToRemove)
    {
        [self removeRepresentation:rep2];
    }

    return [bitmap autorelease];
}

- (NSColor*)colorAtX:(NSInteger)x y:(NSInteger)y
{
    return [self.bitmap colorAtX:x y:y];
}

- (void)setColor:(NSColor*)color atX:(NSInteger)x y:(NSInteger)y
{
    [self.bitmap setColor:color atX:x y:y];
}

NSImage+Extra.m (NSImage Category)

- (CGImageRef)CGImage
{
    return [self CGImageForProposedRect:NULL context:[NSGraphicsContext currentContext] hints:nil];
}

Usage

- (IBAction)load:(id)sender
{
    NSOpenPanel* openDlg = [NSOpenPanel openPanel];

    [openDlg setCanChooseFiles:YES];

    [openDlg setCanChooseDirectories:YES];

    if ( [openDlg runModalForDirectory:nil file:nil] == NSOKButton )
    {
        NSArray* files = [openDlg filenames];

        for( int i = 0; i < [files count]; i++ )
        {
            NSString* fileName = [files objectAtIndex:i];
            BitmapImage *image = [[BitmapImage alloc] initWithContentsOfFile:fileName];

            imageView.image = image;
        }
    }
}

- (IBAction)colorize:(id)sender
{
    float width = imageView.image.size.width;
    float height = imageView.image.size.height;

    BitmapImage *img = (BitmapImage*)imageView.image;

    NSColor *newColor = [img colorAtX:1 y:1];

    for (int x = 0; x <= width; x++)
    {
        for (int y = 0; y <= height; y++)
        {
            if ([img colorAtX:x y:y] == newColor)
            {
                [img setColor:[NSColor redColor] atX:x y:y];
            }
        }
    }
    [imageView setNeedsDisplay:YES];
}

- (IBAction)rotate:(id)sender
{
    BitmapImage *img = (BitmapImage*)imageView.image;
    BitmapImage *newImg = [img rotate90DegreesClockwise:NO];
    imageView.image = newImg;
}

Edit (24/04/2013)

I have changed the following code:

- (RGBColor)colorAtX:(NSInteger)x y:(NSInteger)y
{
    NSUInteger components[4];
    [self.bitmap getPixel:components atX:x y:y];
    //NSLog(@"R: %ld, G:%ld, B:%ld", components[0], components[1], components[2]);

    RGBColor color = {components[0], components[1], components[2]};

    return color;
}

- (BOOL)color:(RGBColor)a isEqualToColor:(RGBColor)b
{
    return ((a.red == b.red) && (a.green == b.green) && (a.blue == b.blue));
}

- (void)setColor:(RGBColor)color atX:(NSUInteger)x y:(NSUInteger)y
{
    NSUInteger components[4] = {(NSUInteger)color.red, (NSUInteger)color.green, (NSUInteger)color.blue, 255};

    //NSLog(@"R: %ld, G: %ld, B: %ld", components[0], components[1], components[2]);

    [self.bitmap setPixel:components atX:x y:y];
}

- (IBAction)colorize:(id)sender
{
    float width = imageView.image.size.width;
    float height = imageView.image.size.height;

    BitmapImage *img = (BitmapImage*)imageView.image;

    RGBColor oldColor = [img colorAtX:0 y:0];
    RGBColor newColor;// = {255, 0, 0};
    newColor.red = 255;
    newColor.green = 0;
    newColor.blue = 0;

    for (int x = 0; x <= width; x++)
    {
        for (int y = 0; y <= height; y++)
        {
            if ([img color:[img colorAtX:x y:y] isEqualToColor:oldColor])
            {
                [img setColor:newColor atX:x y:y];
            }
        }
    }
    [imageView setNeedsDisplay:YES];
}

But now it changes the pixels to red the first time and then blue the second time the colorize method is called.

Edit 2 (24/04/2013)

The following code fixes it. It was because the rotation code was adding an alpha channel to the NSBitmapImageRep.

- (RGBColor)colorAtX:(NSInteger)x y:(NSInteger)y
{
    if (self.bitmap.hasAlpha)
    {
        NSUInteger components[4];
        [self.bitmap getPixel:components atX:x y:y];
        RGBColor color = {components[1], components[2], components[3]};
        return color;
    }
    else
    {
        NSUInteger components[3];
        [self.bitmap getPixel:components atX:x y:y];
        RGBColor color = {components[0], components[1], components[2]};
        return color;
    }
}

- (void)setColor:(RGBColor)color atX:(NSUInteger)x y:(NSUInteger)y
{
    if (self.bitmap.hasAlpha)
    {
        NSUInteger components[4] = {255, (NSUInteger)color.red, (NSUInteger)color.green, (NSUInteger)color.blue};
        [self.bitmap setPixel:components atX:x y:y];
    }
    else
    {
        NSUInteger components[3] = {color.red, color.green, color.blue};
        [self.bitmap setPixel:components atX:x y:y];
    }
}