running opencv code with matlab via mex fails while on VisualStudio it works

Question

I want to extract some harriscorners from an image and get FREAK descriptors. Here is how I try to do it: (The passed variables are globally defined.)

void computeFeatures(cv::Mat &src, std::vector<cv::KeyPoint> &keypoints, cv::Mat &desc ) {
    cv::Mat featureSpace;
    featureSpace = cv::Mat::zeros( src.size(), CV_32FC1 );

    //- Detector parameters
    int blockSize = 3;
    int apertureSize = 3;
    double k = 0.04;

    //- Detecting corners
    cornerHarris( src, featureSpace, blockSize, apertureSize, k, cv::BORDER_DEFAULT );

    //- Thresholding featureSpace
    keypoints.clear();
    nonMaximumSuppression(featureSpace, keypoints, param.nms_n); 

    //- compute FREAK-descriptor
    cv::FREAK freak(false, false, 22.0f, 4); 
    freak.compute(src, keypoints, desc);
}

I can compile it with Visual Studio 12 as well as Matlab R2013b via mex. When I run it as "stand alone" (.exe) it works just fine. When I try to execute it via Matlab it fails with this message:

A buffer overrun has occurred in MATLAB.exe which has corrupted the program's internal state. Press Break to debug the program or Continue to terminate the program.

I mexed with the debug option '-g' and attached VisualStudio to Matlab to be able to get closer to the error: After nonMaximumSuppression() the size of keypoints is 233 when I jump into freak.compute() the size is suddenly 83 with "random" values stored. The actual error is then in KeyPointsFilter::runByKeypointSize when keypoints should be erased.

in keypoint.cpp line 256:

void KeyPointsFilter::runByKeypointSize( vector<KeyPoint>& keypoints, float minSize, float maxSize )
{
    CV_Assert( minSize >= 0 );
    CV_Assert( maxSize >= 0);
    CV_Assert( minSize <= maxSize );

    keypoints.erase( std::remove_if(keypoints.begin(), keypoints.end(), SizePredicate(minSize, maxSize)),
                     keypoints.end() );
}

Is there some error I'm making with passing the keyPoint-vector? Has anybody run into a similar problem?

EDIT:

Here is the mex-file with the additional library "opencv_matlab.hpp" taken from MatlabCentral

#include "opencv_matlab.hpp"

void mexFunction (int nlhs,mxArray *plhs[],int nrhs,const mxArray *prhs[]) {

    // read command
    char command[128];
    mxGetString(prhs[0],command,128);

    if (!strcmp(command,"push") || !strcmp(command,"replace")) {

        // check arguments
        if (nrhs!=1+1 && nrhs!=1+2)
          mexErrMsgTxt("1 or 2 inputs required (I1=left image,I2=right image).");
        if (!mxIsUint8(prhs[1]) || mxGetNumberOfDimensions(prhs[1])!=2)
          mexErrMsgTxt("Input I1 (left image) must be a uint8_t matrix.");

        // determine input/output image properties
        const int *dims1    = mxGetDimensions(prhs[1]);
        const int nDims1    = mxGetNumberOfDimensions(prhs[1]);
        const int rows1     = dims1[0];
        const int cols1     = dims1[1];
        const int channels1 = (nDims1 == 3 ? dims1[2] : 1);

        // Allocate, copy, and convert the input image
        // @note: input is double
        cv::Mat I1_ = cv::Mat::zeros(cv::Size(cols1, rows1), CV_8UC(channels1));
        om::copyMatrixToOpencv<uchar>((unsigned char*)mxGetPr(prhs[1]), I1_);

        // push back single image
        if (nrhs==1+1) {

          // compute features and put them to ring buffer
            pushBack(I1_,!strcmp(command,"replace"));

        // push back stereo image pair
        } else {

          if (!mxIsUint8(prhs[2]) || mxGetNumberOfDimensions(prhs[2])!=2)
            mexErrMsgTxt("Input I2 (right image) must be a uint8_t matrix.");

          // determine input/output image properties
          const int *dims2    = mxGetDimensions(prhs[2]);
          const int nDims2    = mxGetNumberOfDimensions(prhs[2]);
          const int rows2     = dims2[0];
          const int cols2     = dims2[1];
          const int channels2 = (nDims2 == 3 ? dims2[2] : 1);

          // Allocate, copy, and convert the input image
          // @note: input is double
          cv::Mat I2_ = cv::Mat::zeros(cv::Size(cols2, rows2), CV_8UC(channels2));
          om::copyMatrixToOpencv<uchar>((unsigned char*)mxGetPr(prhs[2]), I2_);

          // check image size
          if (dims1_[0]!=dims2_[0] || dims1_[1]!=dims2_[1])
            mexErrMsgTxt("Input I1 and I2 must be images of same size.");

          // compute features and put them to ring buffer
          pushBack(I1_,I2_,!strcmp(command,"replace"));
        }
    }else {
    mexPrintf("Unknown command: %s\n",command);
  }
}

And here is an additional part of the main cpp project.

std::vector<cv::KeyPoint> k1c1, k2c1, k1p1, k2p1; //KeyPoints
cv::Mat d1c1, d2c1, d1p1, d2p1; //descriptors

void pushBack (cv::Mat &I1,cv::Mat &I2,const bool replace) {
    // sanity check
    if (I1.empty()) {
        cerr << "ERROR: Image empty!" << endl;
        return;
    }

    if (replace) {
        //if (!k1c1.empty()) 
        k1c1.clear(); k2c1.clear();
        d1c1.release(); d2c1.release();
    } else {
        k1p1.clear(); k2p1.clear();
        d1p1.release(); d2p1.release();

        k1p1 = k1c1; k2p1 = k2c1;
        d1c1.copyTo(d1p1); d2c1.copyTo(d2p1);

        k1c1.clear(); k2c1.clear();
        d1c1.release(); d2c1.release();
    }

    // compute new features for current frame
    computeFeatures(I1,k1c1,d1c1); 
    if (!I2.empty())
        computeFeatures(I2,k2c1,d2c1);
}

And here is how I call the mex-file from Matlab

I1p = imread('\I1.bmp');
I2p = imread('\I2.bmp');
harris_freak('push',I1p,I2p);

Hope this helps...

Answer 1

I hope this is the correct way to give an answer to my own question.

After a couple of days I found kind of a work around. Instead of building the mex file in Matlab, which gives the above mentioned error, I built it in Visual Studio with instructions taken from here. Now everything works just fine.

It kind of bothers me to not know how to do it with matlab, but hey, maybe someone still has an idea.

Thanks to the commenters for taking the time to look through my question!

Answer 2

If you have the Computer Vision System Toolbox then you do not need mex. It includes the detectHarrisFeatures function for detecting Harris corners, and the extractFeatures function, which can compute FREAK descriptors.