Pixel coordinates to 3D line (opencv)

Question

I have an image displayed on screen which is undistorted via cvInitUndistortMap & cvRemap (having done camera calibration), and the user clicks on a feature in the image. So I have the (u,v) pixel coordinates of the feature, and I also have the intrinsic matrix and the distortion matrix.

What I'm looking for is the equation of the 3D line in camera/real-world coordinates on which the feature the user clicked must lie. I already have the perpendicular distance between the camera's image plane and the feature, so I can combine that with the aforementioned equation to give me the (X,Y,Z) coordinate of the feature in space.

Sounds easy (inverse intrinsic matrix or something?) but I can't find step-by-step instructions anywhere. C++ or C# code preferred.

Answer 1

This is a bit old question but still might be usefull for someone. All lines go through the point (0,0,0), so:

line.x0 = 0; line.y0 = 0; line.z0 = 0;

direction vector is as follows: line.A = (u/fx) - (cx/fx); line.B = (v/fy) - (cy/fy); line.C = 1;

cx,cy,fx,fy are parameters from camera matrix. Equations are explained in "Learning OpenCv" book.