OpenCV: howto use mask parameter for feature point detection (SURF)

Question

I want to limit a SurfFeatureDetector to a set of regions (mask). For a test I define only a single mask:

Mat srcImage; //RGB source image
Mat mask = Mat::zeros(srcImage.size(), srcImage.type());
Mat roi(mask, cv::Rect(10,10,100,100));
roi = Scalar(255, 255, 255);
SurfFeatureDetector detector();
std::vector<KeyPoint> keypoints;
detector.detect(srcImage, keypoints, roi); // crash
//detector.detect(srcImage, keypoints); // does not crash

When I pass the "roi" as the mask I get this error:

OpenCV Error: Assertion failed (mask.empty() || (mask.type() == CV_8UC1 && mask.size() == image.size())) in detect, file /Users/ux/Downloads/OpenCV-iOS/OpenCV-iOS/../opencv-svn/modules/features2d/src/detectors.cpp, line 63

What is wrong with this? How can I correctly pass a mask to the SurfFeatureDetector's "detect" method?

Regards,

Answer 1

Two things about the mask.

• the mask should be a 1-channel matrix of 8-bit unsigned chars, which translates to opencv type CV_8U. In your case the mask is of type srcImage.type(), which is a 3-channel matrix
• you are passing roi to the detector but you should be passing mask. When you are making changes to roi, you are also changing mask.

the following should work

Mat srcImage; //RGB source image
Mat mask = Mat::zeros(srcImage.size(), CV_8U);  // type of mask is CV_8U
// roi is a sub-image of mask specified by cv::Rect object
Mat roi(mask, cv::Rect(10,10,100,100));
// we set elements in roi region of the mask to 255 
roi = Scalar(255);  
SurfFeatureDetector detector();
std::vector<KeyPoint> keypoints;
detector.detect(srcImage, keypoints, mask);     // passing `mask` as a parameter

Answer 2

I tacked your ROI code onto some existing code I was working on, with the following changes it worked for me

cv::Mat mask = cv::Mat::zeros(frame.size(), CV_8UC1);  //NOTE: using the type explicitly
cv::Mat roi(mask, cv::Rect(10,10,100,100));
roi = cv::Scalar(255, 255, 255);

//SURF feature detection
const int minHessian = 400;
cv::SurfFeatureDetector detector(minHessian);
std::vector<cv::KeyPoint> keypoints;
detector.detect(frame, keypoints, mask);              //NOTE: using mask here, NOT roi
cv::Mat img_keypoints; 
drawKeypoints(frame, keypoints, img_keypoints, cv::Scalar::all(-1), cv::DrawMatchesFlags::DEFAULT);
cv::imshow("input image + Keypoints", img_keypoints);
cv::waitKey(0);

Without the changes to the type and the use of mask instead of roi as your mask, I'd get a runtime error as well. This makes sense, as the detect method wants a mask -- it should be the same size as the original image, and roi isn't (it's a 100x100 rectangle). To see this visually, try displaying the mask and the roi

cv::imshow("Mask", mask);
cv::waitKey(0);

cv::imshow("ROI", roi);
cv::waitKey(0);

The type has to match also; the mask should be single channel, while your image type is likely of type 16, which maps to CV_8UC3, a triple channel image

Answer 3

If you are looking to apply the same for irregular mask then:

Mat& obtainIregularROI(Mat& origImag, Point2f topLeft, Point2f topRight, Point2f botLeft, Point2f botRight){

        static Mat black(origImag.rows, origImag.cols, origImag.type(), cv::Scalar::all(0));
        Mat mask(origImag.rows, origImag.cols, CV_8UC1, cv::Scalar(0));
        vector< vector<Point> >  co_ordinates;
        co_ordinates.push_back(vector<Point>());
        co_ordinates[0].push_back(topLeft);
        co_ordinates[0].push_back(botLeft);
        co_ordinates[0].push_back(botRight);
        co_ordinates[0].push_back(topRight);
        drawContours( mask,co_ordinates,0, Scalar(255),CV_FILLED, 8 );

       // origImag.copyTo(black,mask);
        //BasicAlgo::getInstance()->writeImage(black);
        return mask;  // returning the mask only
    }

Then as usual, generate SIFT/SURF/... pointer

// Create smart pointer for SIFT feature detector.

Ptr<FeatureDetector> SIFT_FeatureDetector = FeatureDetector::create("SIFT");
vector<KeyPoint> SIFT_Keypoints;
vector<KeyPoint> SIFT_KeypointsRotated; 
Mat maskedImg = ImageDeformationOperations::getInstance()->obtainIregularROI( rotatedImg,rotTopLeft,rotTopRight,rotBotLeft,rotBotRight);
SIFT_FeatureDetector->detect(rotatedImg, SIFT_KeypointsRotated, maskedImg);
Mat outputSIFTKeyPt;
drawKeypoints(rotatedImg, SIFT_KeypointsRotated, outputSIFTKeyPt, keypointColor, DrawMatchesFlags::DEFAULT);