Rectangle detection in image

Question

I'd like to program a detection of a rectangular sheet of paper which doesn't absolutely need to be perfectly straight on each side as I may take a picture of it "in the air" which means the single sides of the paper might get distorted a bit.

The app (iOs and android) CamScanner does this very very good and Im wondering how this might be implemented. First of all I thought of doing:

• smoothing / noise reduction
• Edge detection (canny etc) OR thresholding (global / adaptive)
• Hough Transformation
• Detecting lines (only vertically / horizontally allowed)
• Calculate the intercept point of 4 found lines

But this gives me much problems with different types of images. And I'm wondering if there's maybe a better approach in directly detecting a rectangular-like shape in an image and if so, if maybe camscanner does implement it like this as well!?

Here are some images taken in CamScanner. These ones are detected quite nicely even though in a) the side is distorted (but the corner still gets shown in the overlay but doesnt really fit the corner of the white paper) and in b) the background is pretty close to the actual paper but it still gets recognized correctly:

It even gets the rotated pictures correctly:

And when Im inserting some testing errors, it fails but at least detects some of the contour, but always try to detect it as a rectangle:

And here it fails completely:

I suppose in the last three examples, if it would do hough transformation, it could have detected at least two of the four sides of the rectangle.

Any ideas and tips? Thanks a lot in advance

Answer 1

OpenCV framework may help your problem. Also, you can look to this document for the Android platform.

The full source code is available on Github.