projective geometry: how do I turn a projection of a rectangle in 3D into a 2D view

Question

So the problem is that I have a 3D projection of a rectangle that I want to turn into 2D. That is I have a photo of a sheet of paper laying on a table which I want to transform into a 2D view of that sheet. So what I need is to get an undistorted 2D image by reverting all the 3D/projection transformations and getting a plain view of the sheet from the top. I happened to find some directions on the subject but they don't suggest an immediate instruction on how this can be achieved. It would be really helpful to get a step-by-step instruction of what needs to be done. Or, alternatively, a link to a resource that breaks it down to little details.

Answer 1

You need more information in order to do that. For example the size of the sheet of paper. Let's say you have it.

What you need to learn about is called "homography". This is basically the following situation:

You have the same planar surface (your sheet of paper) and you take a picture of it from two different cameras (Say one is the actual image that you have and the other is the one that you want to obtain - the one with the camera exactly above the sheet of paper).

There exists a transformation from 2D space of one image to 2D space of the other image (homography) and your goal is to find it. Once you find it you just apply it to your image to get the top view.

In order to find the homography matrix you need (at least) 4 points, whose coordinate you know in BOTH images.

An obvious choice for those points are the vertices of the sheet of paper of course. In the image that you have you can locate those by hand. In the target image, you can pick those such that the sheet is centered it (0,0), knowing the dimensions of it.

There is plenty of information about the homography matrix from 4 points online. This is just one of the first I came upon, so there must be better sources out there :)

Note that most often these computation are done in the 2D projective space, as this is a projective transformation.

Answer 2

Petar's answer would be correct if you knew nothing about the shape of the paper – if the paper could be any arbitrary quadrilateral. But since the paper is a rectangle, this constrains the problem enough to determine the homography without needing to know the aspect ratio of the rectangle.

See section 4 of Microsoft Research's "Whiteboard It!" paper for details on how to do this.

Answer 3

If you do the correct 3D --> 2D transform of the vertices, it should work fine. The starting point is to research DaVinci's model for 3D point --> 2D point projective geometry.