Perspective correction of UIImage from Points

Question

I'm working on a app where I'll let the user take a picture e.g of a business card or photograph.

The user will then mark the four corners of the object (which they took a picture off) - Like it is seen in a lot of document/image/business card scanning apps:

My question is how do i crop and fix the perspective according to these four points? I've been searching for days and looked at several image proccessing libraries without any luck.

Any one who can point me in the right direction?

Answer 1

From iOS8+ there is Filter for Core Image called CIPerspectiveCorrection. All you need to do is pass the image and four points.

Also there is one more filter supporting iOS6+ called CIPerspectiveTransform which can be used in similar way (skewing image).

Answer 2

I'm not sure if you've tried the Opencv library yet, but it has a very nice way to deskew an image. I've got here a small snippet that takes an array of corners, your four corners for example, and a final size to map it into.

You can read the man page for warpPerspective on the OpenCV site.

cv::Mat deskew(cv::Mat& capturedFrame, cv::Point2f source_points[], cv::Size finalSize)
{
    cv::Point2f dest_points[4];

    // Output of deskew operation has same color space as source frame, but
    // is proportional to the area the document occupied; this is to reduce
    // blur effects from a scaling component.
    cv::Mat deskewedMat = cv::Mat(finalSize, capturedFrame.type());

    cv::Size s = capturedFrame.size();

    // Deskew to full output image corners
    dest_points[0] = cv::Point2f(0,s.height); // lower left
    dest_points[1] = cv::Point2f(0,0);        // upper left
    dest_points[2] = cv::Point2f(s.width,0);  // upper right
    dest_points[3] = cv::Point2f(s.width,s.height);  // lower right

    // Build quandrangle "de-skew" transform matrix values
    cv::Mat transform = cv::getPerspectiveTransform( source_points, dest_points  );
    // Apply the deskew transform
    cv::warpPerspective( capturedFrame, deskewedMat, transform, s, cv::INTER_CUBIC );

    return deskewedMat;
}

Answer 3

If this image were loaded in as a texture, it'd be extremely simple to skew it using OpenGL. You'd literally just draw a full-screen quad and use the yellow correction points as the UV coordinate at each point.

Answer 4

I don't know exact solution of your case, but there is approach for trapezoid: http://www.comp.nus.edu.sg/~tants/tsm/TSM_recipe.html - the idea is to continuously build transformation matrix. Theoretically you can add transformation that converts your shape into trapecy.

And there are many questions like this: https://math.stackexchange.com/questions/13404/mapping-irregular-quadrilateral-to-a-rectangle , but I didn't check solutions.