find orientation of pixel patch in affine transformed image with respect to original reference

Question

In part of my project I need to compute orientation of a patch in an affine transformed image. Now my problem is that I don't know how can I find this computed orientation with respect to original un-warped image.

For example a point in warped image is found(100,200). I can extract orientation of this point using 8x8 neighboring pixels. suppose it is 30 degree. the warped image in which the point has found is result of applying transformation on original image with 60 degree in each axis.(pitch,yaw and roll).(This orientation extraction is usually known as descriptor extraction in computer vision)

Now the transformation matrix is known. orientation of the point in the transformed image is known as well. The position of the point wrt reference is known(using inverse transformation). Now I want to know what is the new orientation if this point(100,200) goes to reference frame(e.g. 150,250). in another word, what is the new orientation with respect to reference image.

I know this is straight forward to solve if we just have roll angle rotation(60 degree). in this case the orientation wrt reference frame would be 30+60 = 90 .

I tried implement this using OpenCV:

        cv::Mat rvec1(3,1,CV_32F); // rot vector related to B
        rvec1.at<float>(0,0)=0;
        rvec1.at<float>(1,0)=30*to_RAD;
        rvec1.at<float>(2,0)=0;

        cv::Mat rvec2(3,1,CV_32F); // rot vector related to A
        rvec2.at<float>(0,0)=0;
        rvec2.at<float>(1,0)=60*to_RAD;
        rvec2.at<float>(2,0)=0;

        cv::Mat R_A;
        cv::Mat R_B;

        cv::Rodrigues(rvec1, R_B);
        cv::Rodrigues(rvec2, R_A);

        cv::Mat R_combined= R_B*R_A;

        cv::Mat rvec_result;
        cv::Rodrigues(R_combined,rvec_result);

I want to create rotation Mat A and B using 2 rotation vector. and after multiplying these two I want to convert it into rotation vector.But only thing I get is a runtime error on the last line (cv::Rodrigues(R_combined,rvec_result);)

Thank you for your help in advance.

Answer 1

Ok it sounds like you have two rotation matrices (you can use rodrigues to get them) call them A and B. You want to know how to combine them. Lets say that A represents the orientation of your arrow wrt to the patch and B is the patch wrt to the origin. Lets start with our origin at the center of the patch. A describes the orientation of the arrow. Now we want to rotate the origin by B so that our original reference axis are now at B wrt to the new origin. To do that just do

Combined = [B]*[A];

Update

I don't know why your code would give you an error, it works perfectly for me. Here is the code I ran.

cv::Mat rvec1(3,1,CV_32F); // rot vector related to B
rvec1.at<float>(0,0)=0;
rvec1.at<float>(1,0)=30*M_PI/180;;
rvec1.at<float>(2,0)=0;

cv::Mat rvec2(3,1,CV_32F); // rot vector related to A
rvec2.at<float>(0,0)=0;
rvec2.at<float>(1,0)=60*M_PI/180;;
rvec2.at<float>(2,0)=0;

cv::Mat R_A;
cv::Mat R_B;

cv::Rodrigues(rvec1, R_B);
cv::Rodrigues(rvec2, R_A);

cv::Mat R_combined= R_B*R_A;

cv::Mat rvec_result;
cv::Rodrigues(R_combined,rvec_result);

std::cout << rvec1 << std::endl<<std::endl;
std::cout << rvec2 << std::endl<<std::endl;
std::cout << R_A << std::endl<<std::endl;
std::cout << R_B << std::endl<<std::endl;
std::cout << R_combined << std::endl<<std::endl;
std::cout << rvec_result << std::endl<<std::endl;

And here is my output

[0; 0.52359879; 0]

[0; 1.0471976; 0]

[0.49999997, 0, 0.86602545;
0, 1, 0;
-0.86602545, 0, 0.49999997]

[0.86602539, 0, 0.5;
0, 1, 0;
-0.5, 0, 0.86602539]

[-5.9604645e-08, 0, 1;
0, 1, 0;
-1, 0, -5.9604645e-08]

[0; 1.5707964; 0]