Returning similarity between 2 Images with KnnMatch

Question

I have the following sample from the emgu opencv wrapper

HomographyMatrix homography = null;
SURFDetector surfCPU = new SURFDetector(500,true);
        VectorOfKeyPoint modelKeyPoints;
        VectorOfKeyPoint observedKeyPoints;
        Matrix<int> indices;

        Matrix<byte> mask;
        int k = 2;
        double uniquenessThreshold = 0.9; //0.8

        //extract features from the object image
        modelKeyPoints = surfCPU.DetectKeyPointsRaw(modelImage, null);
        Matrix<float> modelDescriptors = surfCPU.ComputeDescriptorsRaw(modelImage, null, modelKeyPoints);
        modelImage.Dispose();                     


        // extract features from the observed image
        observedKeyPoints = surfCPU.DetectKeyPointsRaw(observedImage, null);
        Matrix<float> observedDescriptors = surfCPU.ComputeDescriptorsRaw(observedImage, null, observedKeyPoints);
        observedImage.Dispose();
        BruteForceMatcher<float> matcher = new BruteForceMatcher<float>(DistanceType.L2);
        matcher.Add(modelDescriptors);

        indices = new Matrix<int>(observedDescriptors.Rows, k);
        using (Matrix<float> dist = new Matrix<float>(observedDescriptors.Rows, k))
        {
            matcher.KnnMatch(observedDescriptors, indices, dist, k, null);
            mask = new Matrix<byte>(dist.Rows, 1);
            mask.SetValue(255);
            Features2DToolbox.VoteForUniqueness(dist, uniquenessThreshold, mask);
        }
        //...

after applying the KnnMatch how can i get the number of matched key points and what does the non zero pixle count has to do with getting similarity between 2 images?

Answer 1

1. refer to opencv doc http://www.opencv.org.cn/opencvdoc/2.3.2/html/modules/features2d/doc/common_interfaces_of_descriptor_matchers.html

   void DescriptorMatcher::knnMatch(const Mat& queryDescriptors, vector>& matches, int k, const vector& masks=vector(), bool compactResult=false )

   matches ( which is dist in your code) – Matches. Each matches[i] is k or less matches for the same query descriptor. k – Count of best matches found per each query descriptor or less if a query descriptor has less than k possible matches in total.

2. about similarity(I am not clear about the question), the code is using L2 distance.

   BruteForceMatcher matcher = new BruteForceMatcher(DistanceType.L2);

Answer 2

k-NN will always find a "nearest" neighbor even if the neighbor is far. So if your image have 1000 interest points, they all will be matched to one interest point in the other image, even if the other image have more or less interest points.

The number of matchings is not directly linked to the similarity between 2 images because the matching set contain "good" matchings (inliers) but also many wrong matchings (outliers).

After matching, you should filter the matchings (by using VoteForUniqueness and VoteForSizeAndOrientation) and then try to find an homography matrix. Then, you can count how many matches validate the homography. Theses matchings are the inliers, and the similarity is a little more correlated to the number of inliers.

You can have more detailed informations in this paper :

[Augereau, O., Journet, N., & Domenger, J. P. (2013, February). Semi-structured document image matching and recognition. In DRR.]