OpenCV stereo camera, detect object distance

Question

Im a newbie to OpenCV, Ive built openCV and installed Qt to run the example http://code.google.com/p/opencvstereovision/source/checkout with success. The result is left and right webcam captures and after calibration a rectified image for both and a depth map. I want to do the following. I will click on an object to focus in any of these left or right images. Then I want to detect the corresponding pixel in the other image and calculate the distance to this object. I dont know what to do now and I need guidence. Maybe if I first calculate the corresponding point, I may work the rest out. How can I do this?

Answer 1

It would seem you have a good idea of what's required. This is a bit of a can of worms - so as ever I would recommend getting hold of the famous Hartley & Zisserman book for the canonical explanation. Here's a link to the relevant chapter.

But in brief...

I've not used the opencvstereovision wrapper class directly, but it sounds like it has taken the headache out of calibrating (camera intrinsics and extrinsics) and has calculated rectification via the Homography matrix (H) for planar geometry, or Fundamental matrix (F) for more complex epipolar geometry.

Probably similar to this original post.

What this rectification then means is that it has established the mathematical mapping between the same point in each image.

In a previous answer (from me) you can then do the maths by using the Fundamental matrix to perform triangulation - i.e. calculate distance.

However, note that this distance is only in the image coordinate frame (i.e. in pixels).

What is actually required to perform "real world" measurements (i.e. actual physical distance) is the calculation of the Essential matrix (E) which combines the Fundamental matrix and the cameras intrinsics (K) to, if you will, project the distances into the real world.

Answer 2

class StereoVar
{
StereoVar();
StereoVar(    int levels, double pyrScale,
                                int nIt, int minDisp, int maxDisp,
                                int poly_n, double poly_sigma, float fi,
                                float lambda, int penalization, int cycle,
                                int flags);
virtual ~StereoVar();

virtual void operator()(InputArray left, InputArray right, OutputArray disp);

int        levels;
double    pyrScale;
int        nIt;
int        minDisp;
int        maxDisp;
int        poly_n;
double    poly_sigma;
float    fi;
float    lambda;
int        penalization;
int        cycle;
int        flags;

...
};

Refer to: http://docs.opencv.org/modules/contrib/doc/stereo.html