Low quality aerial stitching with OpenCV

Question

I've been trying to stitch low quality, low resolution (320x180) images, taken by a quadrocopter, in OpenCV recently. Here is what i got:

http://postimg.org/gallery/1rqsycyk/

The pictures taken are almost nadir and as you can see overlapping much. Between each shot is a translation and i tried to place objects on the ground that keep the scene almost planar not to disturb the requirements for a homography. Anyway quite many pictures are not taken into account during the stitching process.

Here another example, (only three images are stitched together):

http://postimg.org/gallery/1wpt3lmo/

I'm using the Surf Featuredetector and believe that the low quality of the images is not working out right for it but i'm not sure about that.

Here's the code i use, i found it on a similar question OpenCV non-rotational image stitching and decided to use it since it worked better than mine:

    Mat pano;
    Stitcher stitcher = Stitcher::createDefault(false);

    stitcher.setWarper(new PlaneWarper());
    stitcher.setFeaturesFinder(new detail::SurfFeaturesFinder(1000,3,4,3,4));
    stitcher.setRegistrationResol(0.1);
    stitcher.setSeamEstimationResol(0.1);
    stitcher.setCompositingResol(1);
    stitcher.setPanoConfidenceThresh(1);
    stitcher.setWaveCorrection(true);
    stitcher.setWaveCorrectKind(detail::WAVE_CORRECT_HORIZ);
    stitcher.setFeaturesMatcher(new detail::BestOf2NearestMatcher(false,0.3));
    stitcher.setBundleAdjuster(new detail::BundleAdjusterRay());
    Stitcher::Status status = Stitcher::ERR_NEED_MORE_IMGS;
    try{
        status = stitcher.stitch(picturesTaken, pano);
    }
    catch(cv::Exception e){}

My other guess is to do the stitching process manually instead of using the Stitcher class, but i'm not sure if it would change much. So the question is: how can i make the stitching process more robust despite of the low quality of the images? Also: does defining ROIs have only an impact on the performance or also on the chance of actual stitching?

Answer 1

The result is not that bad given the quality of the input images!

To improve the quality of the output, I would do (in priority order):

1. an estimation of the camera distortion in order to fix it and make the matching easier
2. perform some histogram or lighting equalization before stitching
3. try to increase the temporal gap between pictures or use another stitcher. A part of the blur in the output is created by the stitcher when merging the images in their overlap areas.

Answer 2

I believe the problem is that you take pictures of textureless regions and it's hard to extract good distinctive keypoints from such smooth regions.

Answer 3

I found this question, which was very helpful for me. I investigate this theme and I have some other tips for you:

About finding similar images:

• You set SURFFeatureFidner with minHessian = 1000. It is really big value (OpenCV suggest 300, I use sometimes 100). This is why there are only matches not all images.
• You set PanoConfidendceThresh to "1", maybe you should set "0.8", it will stitch more images.

About the look of stitched images: There are some other function in pipeline of Stitcher. Try to use:

• stitcher.setSeamFinder(new detail::GraphCutSeamFinder(GraphCutSeamFinderBase::COST_COLOR))

• stitcher.setBlender( detail::Blender::createDefault(Blender::MULTI_BAND, false))

• stitcher.setExposureCompensator (detail::ExposureCompensator::createDefault(ExposureCompensator::GAIN_BLOCKS) )

Maybe this will be helpful for you!