3d reconstruction from calibrated camera images

Question

I am working on 3d reconstruction. And now when I consider a pair of images. I have a set of corresponding points . and I have my camera details. For example I have focus details,Rotation and Translation matrix(4*4). and I want to project my points in 3D(triangulation). So as far as I got to know its pretty straight forward by factor algebra. But I still need to understand it clearly . Does anyone have an idea about how to follow this? I am working to matlab so I need to implement that! I may be again too broad or whatever. But please guide me!

Answer 1

You will need to compute a camera matrix from your known camera attributes; this is a scaling projection matrix that maps a 3D homogeneous point onto your 2D pixel coordinates. (Note that if you want decent accuracy you will also need to measure and systematically rectify your camera's lens distortion, but that is a second-order detail.)

Then, multiply your camera matrix with your pose matrix (the 4x4 rotation/tranlation matrix you mentioned) to get a matrix (say T for total) that projects a point in your chosen 3D coordinate system into the (rectified) pixel coordinates of your camera:

pixel vector Q.x  =  T * point P.x
             Q.y               P.y
             Q.z               P.z 
             Q.w                1

pixel coordinates  =  (Q.x/Q.w, Q.y/Q.w)
"pixel depth" = Q.z/Q.w

You can invert this matrix, and use it to project pixel coordinates into 3D rays by choosing two depths: one at the camera location, and another which is not (best choose either unit depth or the point at infinity). You can't expect your rays to intersect exactly, but you can get a best-fit location based on where the two corresponding rays are closest; more generally, you can treat the intersection of an arbitrary number of rays as a least-squares problem (ideally, conditioned on the error expected from each ray).