How to use points / scene corners of homography as input for ROI in another function

Question

I want to perform object tracking after recognition, currently im able to recognize an object an draw lines around the object.I would now like to use these four points(scene corners) calculated in perspective transform as input for computing ROI and use it for tracking in subsequent frames.

My current implementation

JNIEXPORT jint JNICALL Java_org_opencv_samples_tutorial2_ObjReco_MatchFeatures(JNIEnv*, jobject, jlong addrGray12, jlong addrRgba12,jlong kalmanaddrRgb)
{
Mat& mGr12  = *(Mat*)addrGray12;
Mat& mRgb12 = *(Mat*)addrRgba12;
Mat H = findHomography( obj, scene, CV_RANSAC); // to find homography
perspectiveTransform( obj_corners, scene_corners, H);// obtain scene corners using perspective transform
setrectagnlepoint(scene_corners); // method to set the points 
}

JNIEXPORT void JNICALL Java_org_opencv_samples_tutorial2_LKTrack_LKTracker(JNIEnv*, jobject, jlong LKtrackgray1,jlong LKtracknew2,jlong LKTrackRgba1)
{

vector<Point2f> features;
vector<Point2f> RectRoi;
Mat RoiLK;
Rect rect;
Mat LKprev;
Mat& LKRgba1 = *(Mat*)LKTrackRgba1;
Mat& LKgray2 = *(Mat*)LKtrackgray1;
Mat regiongray;

LKcounter++;
if (LKcounter == 1) // set rect using points from previous method for the first time 
{
RectRoi=getrectagnlepoint(); // get points from previous method to set ROI
rect = Rect(RectRoi[0].x,RectRoi[0].y,(RectRoi[1].x-RectRoi[0].x),(RectRoi[3].y-RectRoi[0].y));
//rect = Rect(100,100,40,40);
}

LKgray2.copyTo(LKprev);
RoiLK= LKprev(rect);

    cv::goodFeaturesToTrack(RoiLK,features,6000,0.6,2,Mat(),2);

  // Update features to your ROI location
    for (i=0; i<features.size();i++)
    {
        features[i].x= features[i].x+RectRoi[0].x;
        features[i].y= features[i].y+RectRoi[0].y;
    }


// code to calculate optical flow

// code to predict new bounding box

}

currently i am facing assertion error when the points of the rectangle is skewed or if the points do not form a perfect rectangle.

OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::Mat::Mat(const cv::Mat&, const Rect&), file /home/reports/ci/slave_desktop/50-SDK/opencv/modules/core/src/matrix.cpp, line 323

Question is, how do i make sure the points from perspective transform form a perfect rectangle and also is there a method to form ROI for a skewed rectangle?. Any guidance on this will be greatly helpful.

Answer 1

OpenCV's ROI are always a straight rectangle.

You can check here how to create a bounding box for your 4 points - basically your 4 points are referred in that tutorial as a "contour".

Relevant code:

approxPolyDP( Mat(contour), contour_poly, 3, true );
boundRect = boundingRect( Mat(contour_poly) );

Answer 2

it looks as if the transform maps the object points to points that are outside the scene region. you would have to check and verify the transformed points.