findHomography with RANSAC wrong outliering

Question

I am doing some object detection using features2d (ORB, SIFT etc)

I am investigating further into homography with RANSAC. I found out that many good points are marked wrongly as outliers.

There are lots of outliers that shouldn't be outliers inside the object (the book)

I have tried to rise threshold up to 10 but the result is pretty the same. Can I do something to improve this situation?

Is there something broke with RANSAC ?

Answer 1

Outliers in this case means that those points don't fit the model, but they don't necessarily have to fall outside your object.

RANSAC finds the best set of points that fits the model, and the rest are considered outliers, but an outlier can perfectly be a point of the object you are tracking that is affected by noise and its detected position is deviated enough from RANSAC's reprojection to be considered an outlier.

You don't have to worry, it is working fine and your object is detected, as shown in your image.

Answer 2

Try to find more features (min 100) by using other feature detector or tune the detector threshold to increase the number. It is also suitable to distribute points equally on the object. You can try goodFeaturesToTrack from OpenCV.

Answer 3

Outliers in this case are points which do not fit in model. RANSAC in Opencv finds planar homography between the matchin points pair. In your case book is a plane and inliers should be inside book. If they are considered as outliers, reasons might be

1. Matches are wrong
2. Redundancy in image features (easy eg. Checkerboard)

In cases where feature points are of 3D object, there is no guarantee of inliers lying inside object. Opencv's findHomography estimates plane homography if matches are given as input in image coordinates