cvCalcOpticalFlowPyrLK not working as expected

Question

I know this is a well documented problem but I didn't manage to find a satisfactory solution online. Here goes.

I am using cvCalcOpticalFlowPyrLK to track motion of feature points. I find the feature points with cvGoodFeaturesToTrack and refine it with cvFindCornerSubPix. I find the feature points in my first frame (reference frame) and use LK to track the movement of these points with respect to the reference frame. I update the points with current frame feature points coordinate with they are found. Heres what I observed:

1) The no. of good feature points found by cvGoodFeaturesToTrack is very little. I tried to find 100 points but I always get less than 10 points. 2) The no. of feature points after 5-6 frames decreased by 50 percent and then another 50 by 5 frames later, and this is when the subject is not in motion. The tracking is patchy in the sense some of the points are correctly tracked but some are way off.

I have seen demo application on youtube or iphone app. The drop off of the no. of feature points from frame to frame is not what I see in my application. So I am suspecting parameters I set might be wrong.

This is how I call the functions:

cvGoodFeaturesToTrack( image, eigen_image, temp_image, corners_point, &corner_count, 0.01(quality level), 3(min distance), 0, 10(block size), 0(use harris), 0.04(k));

cvFindCornerSubPix( image, cornersPoint, corner_count, cvSize(WINDOW_SIZE, WINDOW_SIZE), cvSize(-1, -1), cvTermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.3));

cvCalcOpticalFlowPyrLK(image, currentFrame, rpV->pyramid_images0, rpV->pyramid_images1, cornersPoint, cornersCurrent, corner_count, cvSize(WINDOW_SIZE, WINDOW_SIZE), 10(level), features_found, feature_errors, cvTermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.3), 0);

Another thing is that I am using a greyscale camera (infra red camera). I dont it matters by too much though. I am wondering if I am missing anything important here.

Any form of help is much appreciated.

Thanks, Kelvin

Answer 1

There are a few issues:

• Calling cvFindCornerSubpix does not help if the features you are tracking don't look like corners of a checkerboard.
• Use of a pyramid is appropriate only if the apparent motion is larger than the window size from frame to frame, for a reasonable window size.
• Hard to tell why you are not getting enough good features to track without seeing your imagery. Perhaps it's rather blurry?