homography matrix and image transformation with less keypoints

Question

When homograpgy matrix with RANSAC is calculated with less numbers of matched keypoints(>4), the transformed image with warpPerspertive, is somehow distorted or tilted. when same image is used to warp with more number of matched keypionts, the result are good. I saw a similar question here but it is still unsolved.

Theoritically we need only 4 matched keypoints, but in parctical why it doesn't work? Can anybody please explain or provide some paper which explains the reasons behind.

Thanks

Answer 1

One big part of the problem is that we have limited precision in an image. Many algorithms make use of sub-pixel accuracy, but it is difficult to find the exact location for a wide variety of reasons. Just a few are that visual information is lost by discretizing a scene into pixels, differences in image resolution, camera sensor imperfections and characteristics, lighting changes, etc.

Consider the 1D case. Say we have two points that are supposed to placed at the exact locations 0 and 20.5, but instead are found at 0 and 20. This is a 2.5% error. If those points were used to transform an 1D image of size 500, some pixels can be off by 12.5 pixels. These errors have a very large effect in parts of the image far away from the correspondences we found. This is called a measurement error.

Obviously if we have one bad correspondence the homography will be bad. These bad matches are called a classification error. In practice we cannot count on any four matches to be perfectly correct that were generated using any of the OpenCV point descriptors. We can (depending on the scene and set up) count on many points to be close to correct.

Take a look at Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography by Fishler and Bolles. It describes an algorithm similar to RANSAC in OpenCV. It discusses briefly the effect of classification and measurement errors. It says that measurement errors typically follow a normal distribution so if we have many matches, measurement errors will have minimal effect. We can also find a way to determine which matches are likely to be a classification error and minimize their effect.