Calling stereoRectify in OpenCV causes exception: invalid frame pointer

Question

For my employer I am comparing the results of an already implemented image rectification method with the results of the corresponding OpenCV implementation. However, an exception is thrown once the OpenCV function is called.

The header of the OpenCV rectification function is

void stereoRectify(InputArray cameraMatrix1, InputArray distCoeffs1, 
                   InputArray cameraMatrix2, InputArray distCoeffs2, 
                   Size imageSize, InputArray R, InputArray T, OutputArray R1, 
                   OutputArray R2, OutputArray P1, OutputArray P2, 
                   OutputArray Q, int flags=CALIB_ZERO_DISPARITY, 
                   double alpha=-1, Size newImageSize=Size(), 
                   Rect* validPixROI1=0, Rect* validPixROI2=0);

As InputArray and OutputArray I used objects of type

cv::Mat

. Since the calibration of the cameras is already known, I initialized the input matrices manually with the correct values. The matrices have the following sizes in accordance with the corresponding documentation page:

cv::Mat cameraMatrix1; // 3x3 matrix
cv::Mat distCoeffs1;   // 5x1 matrix for five distortion coefficients
cv::Mat cameraMatrix2; // 3x3 matrix
cv::Mat distCoeffs2;   // 5x1 matrix
cv::Mat R;             // 3x3 matrix, rotation left to right camera
cv::Mat T;             // 4x1 matrix, translation left to right proj. center

I initialized the matrices like this:

T = cv::Mat::zeros(4, 1, CV_64F);
T.at<double>(0, 0) = proj_center_right.x - proj_center_left.x;
T.at<double>(1, 0) = proj_center_right.y - proj_center_left.y;
T.at<double>(2, 0) = proj_center_right.z - proj_center_left.z;

For all matrices I used CV_64F as value type.

I printed the content of the matrices on the console, to verify, that all values are set correctly (rounded):

cameraMatrix1:
| 6654; 0, 1231 |
| 0; 6654; 1037 |
| 0; 0; 1 |

distCoeffs1:
| -4.57e-009; 5.94e-017; 3.68e-008; -3.46e-008; 6.37e-023 |

cameraMatrix2:
| 6689; 0, 1249 |
| 0; 6689; 991 |
| 0; 0; 1|

distCoeffs2:
| -4.72e-009; 2.88e-016; 6.2e-008; -8.74e-008; -8.18e-024 |

R:
| 0.87; -0.003, -0.46 |
| 0.001; 0.999; -0.003 |
| 0.46; 0.002; 0.89 |

T:
| 228; 0; 0; 0 |

Everything seems correct to me so far. Further, I initialized the output matrices as identity matrices (using cv::Mat::eye(...)), with the following sizes:

cv::Mat R1; // 3x3 matrix
cv::Mat R2; // 3x3 matrix
cv::Mat P1; // 3x4 matrix
cv::Mat P2; // 3x4 matrix
cv::Mat Q;  // 4x4 matrix

Finally the required cv::Size object is set to width 2448 and height 2050 (size of the images acquired by the cameras). Once I pass the parameters to OpenCV as

cv::stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imgSize, R, T, R1, R2, P1, P2, Q);

, the program crashes. The error message on the console states

opencv_core248, void cdecl cv::error(class cv::Exception const & ptr64) +0x152 (invalid frame pointer)

Since all matrices and the cv::Size object are initialized correctly, I do not see, what might be wrong. For any suggestions, I am thankful.

Answer 1

your code initially crashed for me in gemm(), changing T to a 3x1 vec seemed to help:

// Mat_<double> used here for easy << initialization
cv::Mat_<double> cameraMatrix1(3,3); // 3x3 matrix
cv::Mat_<double> distCoeffs1(5,1);   // 5x1 matrix for five distortion coefficients
cv::Mat_<double> cameraMatrix2(3,3); // 3x3 matrix
cv::Mat_<double> distCoeffs2(5,1);   // 5x1 matrix
cv::Mat_<double> R(3,3);             // 3x3 matrix, rotation left to right camera
cv::Mat_<double> T(3,1);             // * 3 * x1 matrix, translation left to right proj. center
// ^^ that's the main diff to your code, (3,1) instead of (4,1)

cameraMatrix1 << 6654, 0, 1231, 0, 6654, 1037, 0, 0, 1;
cameraMatrix2 << 6689, 0, 1249, 0, 6689, 991, 0, 0, 1;
distCoeffs1   << -4.57e-009, 5.94e-017, 3.68e-008, -3.46e-008, 6.37e-023;
distCoeffs2   << -4.72e-009, 2.88e-016, 6.2e-008, -8.74e-008, -8.18e-024;
R <<  0.87, -0.003, -0.46, 0.001, 0.999, -0.003, 0.46, 0.002, 0.89;
T << 228, 0, 0;

cv::Mat R1,R2,P1,P2,Q;   // you're safe to leave OutpuArrays empty !
Size imgSize(3000,3000); // wild guess from you cameramat ( not that it matters )

cv::stereoRectify(cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, imgSize, R, T, R1, R2, P1, P2, Q);

cerr << "Q" << Q << endl;