First of all you should make sure you understand the basic concepts of epipolar geometry, especially the concept of an epipolar line.
In a nutshell: Say you have a 3D point P
that projects to a 2D point q
in the image of a camera A
. Now you have a second camera, call it B
, and you want to find the image of P
in B
. Epipolar geometry tells you that the possible locations of the image of P
in B
given q
are limited to a line, called the epipolar line. It also tells you that (and how) you can compute this line from q
and the calibration of your cameras, using what is called the Fundamental matrix.
For your problem this has the following implications:
Let q
be a point from list L0
.
- If there is a single point in your list
L1
which is on the epipolar line ofq
in the second image, then this is the correct correspondence forq
. - If there are several points on the epipolar line, then your problem cannot be solved with the given information. In this case you need to make heuristic assumptions about the distribution of your points in 3D, since you get one possible 3D point for each pairing of
q
with a point on the epipolar line.
If your cameras are only offset along the X axis, and if they are identically oriented (i.e. the image planes are parallel), they it is your lucky day: In this special camera configuration, the epipolar lines are horizontal, i.e. for a point (x,y)
the epipolar line is the line of all points with arbitrary X coordinate and Y coordinate y
.
Note that in practice you also face the problem that points are unlikely to be exactly on epipolar lines due to measurement errors and numerical issues.