Projection of 3D Coordinates onto a 2D image with known points

Question

I'm writing a .NET program that allows a user to register an image by identifying specific points on an image and then specifying the real world coordinates associated with each of those points.

http://www.ironbyte.ca/temp/mountain.jpg

The image registration process also requires the user to specify the coordinates of the camera.

What I'd the like to be able to do after the image is registered is draw other points on the image based on their real-world coordinates.

I've done a great deal of reading on perspective projections but I'm struggling to get things working. I must admit that my math skills are not what they should be which is part of the struggle. Where I am getting stuck is trying to determine focal length and distance to the display surface:

Referred to as the Viewer's Position (e [x,y,z]) in this article: http://en.wikipedia.org/wiki/3D_projection#Perspective_projection

I've also been referring to this article as well: http://www.shotlink.com/Tour/WebTemplate/shotlinknew.nsf/2c47cc31e412bc4985256e6e00287832/c1743b40acf6aa03852575b7007122b0/$FILE/Plotting%203D%20ShotLink%20Data%20on%202D%20Images.pdf which extracts the focal length from the field of view, which appears to be know beforehand, but is not in my case.

So I guess my question then is, is there a way I can work in reverse to determine focal length and/or field of view based on the position of the known points on my image? Or am I looking at this the wrong way and maybe there is an easier way to accomplish the end goal?

Answer 1

I was able to solve this problem using an implementation of the Tsai algorithm (http://en.wikipedia.org/wiki/Camera_resectioning#Algorithms) which can compute a projection matrix using a minimum of 4 known points. Basically, I allow the user to specify the world coordinates of a point and then click on the image to specify the image coordinates. The algorithm uses these mappings (the more mappings, the more accurate the solution is), along with the image width and height to calculate a projection matrix. This projection matrix can then be used to project additional points onto the image using world coordinates.

Answer 2

---

EDIT: I got myself confused by mixing units on the schema. I thus reworked a bit my answer.

---

It sounds feasible to me, if we look at the maths behind the projection.

Here is a not-so-rigorous schema of the situation for the horizontal coordinate (I'm mixing real world coordinates and pixels one to try to illustrate your situation):

With:

• D, one of the points given by the users, with (x,y,z) its projected position with respects to the relative coordinate system defined by the camera (so after applying its translation and rotation)
• E the camera point - origin of the coordinate system described above.
• B the resulting point in your picture plane, with u and v in pixels. The picture plane has for dimensions w x h pixels.
• f the focal length (same unit as for x, y, z...) in pixels, F its value in the real-world unit, and α the horizontal half-angle of view - the values you want to evaluate

---

You can see that the triangles ECD and EBM are similar, so using the Side-Splitter Theorem, we get:

• EM / EC = MB / CD <=> f / z = u / x (we are comparing ratio, so no problem if the left member of the equation uses a real-world unit while the right one uses pixels are real-world values divided by pixels one)

We thus get:

• f = u / x * z

Now if you want α F, I think you'll need to know the dimensions r_x x r_y (real-world unit) of your camera's sensor, since:

• tan(α) = (r_x / 2) / f F = r_x / (w / 2) * f

But as for α, you can get it through:

• tan(α) = (w / 2) / f

---

If you want to do the parallel with the Wikipedia article you're pointing out, we've been using:

Where:

• (d_x,d_y,d_z) = (x,y,z), position of the point in the camera system
• (s_x,s_y) = (w,h), size of your printable surface
• (r_x,r_y,r_z) = (r_x,r_y,f), characteristics of your recording surface
• (b_x,b_y) = (u,v), position on your printable surface