How to Detect corners on image using GPUImageHarrisCornerDetectionFilter

Question

I'm trying to crop an image with a square in it by detecting the corners. The method below attempts to find the corners, but I'm not really sure how to finish the process. Any thoughts?

The optimal function would detect corners, crop the image using those corners, and return the cropped image (the square in the picture).

-(NSArray *)cornersForImage:(UIImage *)inputImage{

    NSMutableArray *results = [NSMutableArray new];

    GPUImagePicture *stillImageSource = [[GPUImagePicture alloc] initWithImage:inputImage];

    GPUImageHarrisCornerDetectionFilter *cornerFilter1 = [[GPUImageHarrisCornerDetectionFilter alloc] init];
    [cornerFilter1 setThreshold:0.1f];
    [cornerFilter1 forceProcessingAtSize:self.displayImage.frame.size];

    [stillImageSource addTarget:cornerFilter1];
    [stillImageSource processImage];

    [cornerFilter1 setCornersDetectedBlock:^(GLfloat *, NSUInteger, CMTime) {

        //get corners?
    }];
    return [NSArray arrayWithArray:results];
}

Answer 1

You're going to get your corners back in an array:

[(GPUImageHarrisCornerDetectionFilter *)filter setCornersDetectedBlock:^(GLfloat* cornerArray, NSUInteger cornersDetected, CMTime frameTime) {
}];

where cornersArray in the above is an array of pairs of GLfloat values that describe the normalized X and Y coordinates for each corner (the size of the array is cornersDetected * 2). These values are normalized to the range 0.0-1.0.

If the rectangle you're cropping is a rectangle aligned with the overall image, you can use a GPUImageCropFilter to crop out the rectangle of interest, based on those normalized coordinates.

However, you're most likely not going to get back an aligned rectangle. That's where things get a little more complicated, because you'll need to handle both rotation and perspective of your rectangle. I don't currently have anything that does this, although this question, this question, and this one describe math that may be used to generate a transform which could be used to do the projection you need. I've been wanting to get something like this into the framework, but haven't had the time to work out the math on the proper transform to apply to map from one arbitrary quadrilateral to another.

Answer 2

Can you post the docs on the GPUImageHarrisCornerDetectionFilter, and specifically the setCornersDetectedBlock method?

Note that your code probably won't work as you've written it. It sounds to me like the filter is designed to run asynchronously, so after you call setCornersDetectedBlock it will return before the results are available. You probably put code in the block that processes the results of the corner detection.