OpenCV calibrateCamera doesn't converge to correct solution (projector calibration)

https://stackoverflow.com/questions/23568696

19-07-2023
|

题

I'm trying to use cv::calibrateCamera to calibrate a camera+projector system. While it seems to work sometimes, in many cases the estimated solution seems to be pretty far from the correct one.

Here is a code sample which shows the problem (all data included - therefore it's quite big)

#include <opencv2/core/core.hpp>
#include <opencv2/calib3d/calib3d.hpp>

#include <iostream>
#include <vector>

cv::Mat calculateTransformation(cv::Mat translationVec, cv::Mat rotationVec) {
    cv::Mat rotationMat3x3;
    cv::Rodrigues(rotationVec, rotationMat3x3);
    cv::Mat rotationMat = cv::Mat::eye(4, 4, CV_64F);
    rotationMat3x3.copyTo(rotationMat(cv::Rect(0, 0, 3, 3)));

    cv::Mat translationMat = cv::Mat::eye(4, 4, CV_64F);
    translationVec.copyTo(translationMat(cv::Rect(3, 0, 1, 3)));

    return translationMat * rotationMat;
}

int main() {
    std::vector<cv::Point2f> image_points;
    std::vector<cv::Point3f> object_points;

    image_points.push_back(cv::Point2f(400.0f, 940.0f));
    image_points.push_back(cv::Point2f(480.0f, 940.0f));
    image_points.push_back(cv::Point2f(560.0f, 940.0f));
    image_points.push_back(cv::Point2f(640.0f, 940.0f));
    image_points.push_back(cv::Point2f(720.0f, 940.0f));
    image_points.push_back(cv::Point2f(800.0f, 940.0f));
    image_points.push_back(cv::Point2f(880.0f, 940.0f));
    image_points.push_back(cv::Point2f(960.0f, 940.0f));
    image_points.push_back(cv::Point2f(1040.0f, 940.0f));
    image_points.push_back(cv::Point2f(1120.0f, 940.0f));
    image_points.push_back(cv::Point2f(1200.0f, 940.0f));
    image_points.push_back(cv::Point2f(1280.0f, 940.0f));
    image_points.push_back(cv::Point2f(400.0f, 860.0f));
    image_points.push_back(cv::Point2f(480.0f, 860.0f));
    image_points.push_back(cv::Point2f(560.0f, 860.0f));
    image_points.push_back(cv::Point2f(640.0f, 860.0f));
    image_points.push_back(cv::Point2f(720.0f, 860.0f));
    image_points.push_back(cv::Point2f(800.0f, 860.0f));
    image_points.push_back(cv::Point2f(880.0f, 860.0f));
    image_points.push_back(cv::Point2f(960.0f, 860.0f));
    image_points.push_back(cv::Point2f(1040.0f, 860.0f));
    image_points.push_back(cv::Point2f(1120.0f, 860.0f));
    image_points.push_back(cv::Point2f(1200.0f, 860.0f));
    image_points.push_back(cv::Point2f(1280.0f, 860.0f));
    image_points.push_back(cv::Point2f(400.0f, 780.0f));
    image_points.push_back(cv::Point2f(480.0f, 780.0f));
    image_points.push_back(cv::Point2f(560.0f, 780.0f));
    image_points.push_back(cv::Point2f(640.0f, 780.0f));
    image_points.push_back(cv::Point2f(720.0f, 780.0f));
    image_points.push_back(cv::Point2f(800.0f, 780.0f));
    image_points.push_back(cv::Point2f(880.0f, 780.0f));
    image_points.push_back(cv::Point2f(960.0f, 780.0f));
    image_points.push_back(cv::Point2f(1040.0f, 780.0f));
    image_points.push_back(cv::Point2f(1120.0f, 780.0f));
    image_points.push_back(cv::Point2f(1200.0f, 780.0f));
    image_points.push_back(cv::Point2f(1280.0f, 780.0f));
    image_points.push_back(cv::Point2f(400.0f, 700.0f));
    image_points.push_back(cv::Point2f(480.0f, 700.0f));
    image_points.push_back(cv::Point2f(560.0f, 700.0f));
    image_points.push_back(cv::Point2f(640.0f, 700.0f));
    image_points.push_back(cv::Point2f(720.0f, 700.0f));
    image_points.push_back(cv::Point2f(800.0f, 700.0f));
    image_points.push_back(cv::Point2f(880.0f, 700.0f));
    image_points.push_back(cv::Point2f(960.0f, 700.0f));
    image_points.push_back(cv::Point2f(1040.0f, 700.0f));
    image_points.push_back(cv::Point2f(1120.0f, 700.0f));
    image_points.push_back(cv::Point2f(1200.0f, 700.0f));
    image_points.push_back(cv::Point2f(1280.0f, 700.0f));
    image_points.push_back(cv::Point2f(400.0f, 620.0f));
    image_points.push_back(cv::Point2f(480.0f, 620.0f));
    image_points.push_back(cv::Point2f(560.0f, 620.0f));
    image_points.push_back(cv::Point2f(640.0f, 620.0f));
    image_points.push_back(cv::Point2f(720.0f, 620.0f));
    image_points.push_back(cv::Point2f(800.0f, 620.0f));
    image_points.push_back(cv::Point2f(880.0f, 620.0f));
    image_points.push_back(cv::Point2f(960.0f, 620.0f));
    image_points.push_back(cv::Point2f(1120.0f, 620.0f));
    image_points.push_back(cv::Point2f(1040.0f, 620.0f));
    image_points.push_back(cv::Point2f(1200.0f, 620.0f));
    image_points.push_back(cv::Point2f(1280.0f, 620.0f));
    image_points.push_back(cv::Point2f(400.0f, 540.0f));
    image_points.push_back(cv::Point2f(480.0f, 540.0f));
    image_points.push_back(cv::Point2f(560.0f, 540.0f));
    image_points.push_back(cv::Point2f(640.0f, 540.0f));
    image_points.push_back(cv::Point2f(720.0f, 540.0f));
    image_points.push_back(cv::Point2f(800.0f, 540.0f));
    image_points.push_back(cv::Point2f(880.0f, 540.0f));
    image_points.push_back(cv::Point2f(1200.0f, 540.0f));
    image_points.push_back(cv::Point2f(1120.0f, 540.0f));
    image_points.push_back(cv::Point2f(480.0f, 460.0f));
    image_points.push_back(cv::Point2f(400.0f, 460.0f));
    image_points.push_back(cv::Point2f(560.0f, 460.0f));
    image_points.push_back(cv::Point2f(720.0f, 460.0f));
    image_points.push_back(cv::Point2f(640.0f, 460.0f));
    image_points.push_back(cv::Point2f(800.0f, 460.0f));
    image_points.push_back(cv::Point2f(1120.0f, 460.0f));
    image_points.push_back(cv::Point2f(880.0f, 460.0f));
    image_points.push_back(cv::Point2f(1040.0f, 460.0f));
    image_points.push_back(cv::Point2f(1280.0f, 460.0f));
    image_points.push_back(cv::Point2f(1120.0f, 380.0f));
    image_points.push_back(cv::Point2f(1280.0f, 380.0f));
    image_points.push_back(cv::Point2f(1200.0f, 380.0f));
    image_points.push_back(cv::Point2f(1040.0f, 380.0f));
    image_points.push_back(cv::Point2f(880.0f, 380.0f));
    image_points.push_back(cv::Point2f(800.0f, 380.0f));
    image_points.push_back(cv::Point2f(960.0f, 380.0f));
    image_points.push_back(cv::Point2f(640.0f, 380.0f));
    image_points.push_back(cv::Point2f(720.0f, 380.0f));
    image_points.push_back(cv::Point2f(560.0f, 380.0f));
    image_points.push_back(cv::Point2f(400.0f, 380.0f));
    image_points.push_back(cv::Point2f(480.0f, 380.0f));
    image_points.push_back(cv::Point2f(1120.0f, 300.0f));
    image_points.push_back(cv::Point2f(1040.0f, 300.0f));
    image_points.push_back(cv::Point2f(960.0f, 300.0f));
    image_points.push_back(cv::Point2f(880.0f, 300.0f));
    image_points.push_back(cv::Point2f(800.0f, 300.0f));
    image_points.push_back(cv::Point2f(720.0f, 300.0f));
    image_points.push_back(cv::Point2f(640.0f, 300.0f));
    image_points.push_back(cv::Point2f(560.0f, 300.0f));
    image_points.push_back(cv::Point2f(480.0f, 300.0f));
    image_points.push_back(cv::Point2f(400.0f, 300.0f));
    image_points.push_back(cv::Point2f(1280.0f, 220.0f));
    image_points.push_back(cv::Point2f(1200.0f, 220.0f));
    image_points.push_back(cv::Point2f(1120.0f, 220.0f));
    image_points.push_back(cv::Point2f(1040.0f, 220.0f));
    image_points.push_back(cv::Point2f(960.0f, 220.0f));
    image_points.push_back(cv::Point2f(800.0f, 220.0f));
    image_points.push_back(cv::Point2f(720.0f, 220.0f));
    image_points.push_back(cv::Point2f(640.0f, 220.0f));
    image_points.push_back(cv::Point2f(560.0f, 220.0f));
    image_points.push_back(cv::Point2f(480.0f, 220.0f));
    image_points.push_back(cv::Point2f(400.0f, 220.0f));
    image_points.push_back(cv::Point2f(1280.0f, 140.0f));
    image_points.push_back(cv::Point2f(1200.0f, 140.0f));
    image_points.push_back(cv::Point2f(1040.0f, 140.0f));
    image_points.push_back(cv::Point2f(960.0f, 140.0f));
    image_points.push_back(cv::Point2f(800.0f, 140.0f));
    image_points.push_back(cv::Point2f(720.0f, 140.0f));
    image_points.push_back(cv::Point2f(640.0f, 140.0f));
    image_points.push_back(cv::Point2f(560.0f, 140.0f));
    image_points.push_back(cv::Point2f(480.0f, 140.0f));
    image_points.push_back(cv::Point2f(400.0f, 140.0f));
    image_points.push_back(cv::Point2f(960.0f, 540.0f));
    image_points.push_back(cv::Point2f(1040.0f, 540.0f));
    image_points.push_back(cv::Point2f(960.0f, 460.0f));

    object_points.push_back(cv::Point3f(-0.0671159000000000f, -0.0530089000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0572335000000000f, -0.0531021000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0473920000000000f, -0.0531268000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0375422000000000f, -0.0531293000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0277093000000000f, -0.0532232000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0179416000000000f, -0.0532900000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00818986000000000f, -0.0533080000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00159555000000000f, -0.0533734000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0113126000000000f, -0.0534102000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0210113000000000f, -0.0534688000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0307716000000000f, -0.0535300000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0404079000000000f, -0.0535600000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0668703000000000f, -0.0429498000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0569997000000000f, -0.0430325000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0471452000000000f, -0.0430930000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0373059000000000f, -0.0431530000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0274813000000000f, -0.0432117000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0176713000000000f, -0.0432687000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00790218000000000f, -0.0433224000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00190183000000000f, -0.0433762000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0116116000000000f, -0.0434242000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0213028000000000f, -0.0434700000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0310283000000000f, -0.0435404000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0406556000000000f, -0.0436058000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0666877000000000f, -0.0329366000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0568031000000000f, -0.0329860000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0469101000000000f, -0.0330818000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0370300000000000f, -0.0331543000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0272122000000000f, -0.0332004000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0173840000000000f, -0.0332688000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00759575000000000f, -0.0333343000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00215042000000000f, -0.0334201000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0119054000000000f, -0.0334816000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0215901000000000f, -0.0335390000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0313087000000000f, -0.0336208000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0409288000000000f, -0.0336740000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0664698000000000f, -0.0228786000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0565702000000000f, -0.0229386000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0466851000000000f, -0.0230437000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0367625000000000f, -0.0230813000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0269267000000000f, -0.0231839000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0171296000000000f, -0.0232845000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00732099000000000f, -0.0232912000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00244503000000000f, -0.0234120000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0121943000000000f, -0.0235079000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0218997000000000f, -0.0235779000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0316661000000000f, -0.0236490000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0413080000000000f, -0.0237153000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0662403000000000f, -0.0127937000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0563243000000000f, -0.0128856000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0464450000000000f, -0.0129539000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0365046000000000f, -0.0130461000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0266992000000000f, -0.0131572000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0168820000000000f, -0.0132217000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00702858000000000f, -0.0133096000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00273311000000000f, -0.0134175000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0222048000000000f, -0.0135829000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0124779000000000f, -0.0135008000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0319668000000000f, -0.0136414000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0416578000000000f, -0.0137210000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0660230000000000f, -0.00272328000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0560886000000000f, -0.00280125000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0461911000000000f, -0.00292352000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0362558000000000f, -0.00302379000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0264298000000000f, -0.00309841000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0165924000000000f, -0.00321897000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00674314000000000f, -0.00331641000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0322628000000000f, -0.00369330000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0225310000000000f, -0.00360024000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0558388000000000f, 0.00733873000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0657448000000000f, 0.00744612000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0458967000000000f, 0.00723106000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0261183000000000f, 0.00701760000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0359654000000000f, 0.00710082000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0162843000000000f, 0.00688856000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0228789000000000f, 0.00646718000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00641300000000000f, 0.00678188000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0131059000000000f, 0.00657206000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0422672000000000f, 0.00628497000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0231953000000000f, 0.0165035000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0425539000000000f, 0.0162757000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0328968000000000f, 0.0163892000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0133982000000000f, 0.0166424000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00611577000000000f, 0.0168486000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0160094000000000f, 0.0169642000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00368791000000000f, 0.0167334000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0357336000000000f, 0.0171710000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0258655000000000f, 0.0170562000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0456858000000000f, 0.0173090000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0655273000000000f, 0.0175171000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0555765000000000f, 0.0174012000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0235599000000000f, 0.0266047000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0137369000000000f, 0.0267060000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00400251000000000f, 0.0268771000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.00579988000000000f, 0.0269546000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0156666000000000f, 0.0270784000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0255455000000000f, 0.0272245000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0354855000000000f, 0.0273244000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0454115000000000f, 0.0274243000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0553239000000000f, 0.0275921000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0652735000000000f, 0.0276931000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0432836000000000f, 0.0364607000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0335970000000000f, 0.0365698000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0239205000000000f, 0.0367035000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0140970000000000f, 0.0368368000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00436217000000000f, 0.0369936000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0152808000000000f, 0.0372578000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0252343000000000f, 0.0373656000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0351242000000000f, 0.0374961000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0450986000000000f, 0.0376261000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0550835000000000f, 0.0377566000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0649853000000000f, 0.0379110000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0436747000000000f, 0.0466393000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0339311000000000f, 0.0467559000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0144269000000000f, 0.0470150000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00466523000000000f, 0.0471564000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0150279000000000f, 0.0474823000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0248820000000000f, 0.0475522000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0348463000000000f, 0.0476901000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0447968000000000f, 0.0478509000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0547835000000000f, 0.0479670000000000f, 0.0f));
    object_points.push_back(cv::Point3f(-0.0647592000000000f, 0.0481078000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00301477000000000f, -0.00341146000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.0128075000000000f, -0.00350684000000000f, 0.0f));
    object_points.push_back(cv::Point3f(0.00336785000000000f, 0.00670002000000000f, 0.0f));

    std::vector<std::vector<cv::Point2f>> all_image_points;
    all_image_points.push_back(image_points);
    std::vector<std::vector<cv::Point3f>> all_object_points;
    all_object_points.push_back(object_points);

    cv::Mat camera_mat = cv::Mat::eye(3, 3, CV_64F);
    cv::Mat dist_coeffs = cv::Mat::zeros(4, 1, CV_64F);
    std::vector<cv::Mat> rvecs;
    std::vector<cv::Mat> tvecs;

    double reprojection_error = cv::calibrateCamera(all_object_points, all_image_points, cv::Size(1920, 1080), camera_mat, dist_coeffs, rvecs, tvecs);
    std::cout << "Reprojection error: " << reprojection_error << std::endl;

    cv::Mat transformation = calculateTransformation(tvecs.front(), rvecs.front());
    cv::Mat origin = cv::Mat::zeros(4, 1, CV_64F);
    origin.at<double>(3, 0) = 1.0;
    std::cout << "distance to board: " << cv::norm(transformation * origin) << std::endl;

    return 0;
}

The object points are measured in meters
The distance to the board should be roughly 0.64 meters (estimated are 1.7 meters)
The radial distortion parameter k2 has a value above 200 which is kind of weird
I've visualized my input data using matlab to verify that the correspondences are correct
I know that I get more robust solutions with multiple captures but the posted data seem to influence the solution quite heavily. Made two captures which produced a solution which seemed right, added another capture (posted data) which led to a wrong solution.
I guess the data is a bit more noisy then regular calibration data because I can't capture the data directly (it's a projector calibration) but need to use a camera which detects an projected pattern

If someone wants to visualize the data using matlab: I've simpliy used

plot3([image_points(1,:);object_points(1,:) * 10000], [image_points(2,:);object_points(2,:) * -10000], [zeros(1, 125); ones(1, 125)], 'b-')

so you get one line per correspondence between object and image point.

If more information is needed just post a comment.

解决方案

My proposition is to use different non-coplanar rigs (surfaces) in camera calibration process. The algorithm opencv uses definitely need at least two different views to perform good calibration. If it possible, provide several views with different angles (2 is just a starting point, 10 is desirable). I'm not 100% sure now, but I think non-planar requirement is sine qua non condition, namely you cannot obtain good results with coplanar views (because in fact they are different only according to the scale factor so in some words we can say they are isomorphic in terms of projection geometry).

And one comment to projection error - in tested case is around 0.65 what seems to be accurate... But here's the tricky part - because we calibrate the camera using only one rig, so by definition, when opencv re project back points, it has only one surface to reproject to. In other words, the calibration error is small and it is greater than zero only because there is a noise giving point's projections.

许可以下： CC-BY-SA 和归因

不隶属于 StackOverflow