Structure from motion from multiple views

Question

I'm using OpenCV to extract 3D model from multiple views (images). I got to a stage where the inputs are two images and the results are 3D points that reconstruct the 3D model.

I want to combine more than two images for a full reconstruction from all views. So I have a 3D points described images 1-2 and a 3D points described images 2-3. How can I merge the two 3D points arrays to a fully 3D model.

any suggestions?

Answer 1

Assuming you are doing the 3D reconstruction using a stereo matching algorithm, the 3D reconstruction between images 1-2 results in 3D points in the coordinate system of image 1. Similarly, the 3D reconstruction between images 2-3 results in 3D points in the coordinate system of image 2.

Hence you simply have to change the 3D coordinate system of your second point cloud, from the one of image 2 to the one of image 1. This makes use of the rotation matrix R and the translation vector T between images 1-2.

EDIT: Note that this way of merging the two point clouds is very basic, and you could improve accuracy by doing a joint 3D reconstruction using images 1-2-3 at once (e.g. bundle adjustment). I don't think that this approach is available in OpenCV though.

Answer 2

It is not as simple as transforming the second pc to the frame where the first one is at, since the two frames are independent. There is a scale ambiguity. If you are lucky enough, you might see the merger works fine after some bundle adjustment.