Using openCV remap with float*

Question

I'm currently working on a project which involves motion compensation.

At this point I have the input image in cv::Mat input;

cv::resize( input, input_resized, cvSize(64,48), 12, 12, CV_INTER_AREA);
input_resized.convertTo(input_float, CV_32F); //this might be redundant while cv::rezise dst should be 32F.

Now the motion vectors are stored in a 2 channel cv::Mat motion and it's stored like this:

// w -image.width , h-image.height
    X0  Y0  X1   Y1   X2   Y2   ...  Xw-1  Yw-1      
    Xw  Yw  Xw+1 Yw+1 Xw+2 Yw+2 ...  X2w-1 Y2w-1
    ...
    X(h-1)w Y(h-1)w   ............. X(h-1)(w-1)Y(h-1)(w-1)

So if I use :

std::vector<cv::Mat> motionvect(2);
cv::split(motion, motionvect);

I will get X in motionvect(0) and Y in motionvect2(1).

Furthermore I've tried:

std::vector<cv::Mat> rgbChannels(3);
cv::split(input_float, rgbChannels);
cv::remap(rgbChannels[0], outChannels[0], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::remap(rgbChannels[1], outChannels[1], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::remap(rgbChannels[2], outChannels[2], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));

cv::merge(outChannels, input_with_motion);

The results I get after this are very strange and somebody told me it is because of the Y vector. Aparently if X is like this:

1  2  3  4  5
6  7  8  9  10
11 12 13 14 15

Then Y should be :

1  6  11  
2  7  12  
3  8  13
4  9  14
5  10 15

Is that correct? Sort of transpose matrix. If it is true, is there any function that will change the motiovect[2] to look like this or should I do it for each individual component?

Also, for the final step I would need to apply the remap at a different time in my process. Something like this:

Function1 ( (float*) input_float.data , (float*) motion.data, (unsigned char*) out.data );
   Function2 ( float* input, float* R, float* G, float* B);
   Function3 ( float* R, float* G, float* B, float* W, float* Z);

where R, G, B, W, Z are float* types and they are allocated with

 R = (float*) malloc (frame_size * sizeof(float) );

I need to apply motion compensation to float* A and float* B, but these are not allowed inputs to remap. They have to be InputArray and I can't find any method to convert float* to any accepted InputArray type.

I know I'm not very organized in thoughts, but I hope you'll understand at least one of my questions. 1. How to apply remap if I have the motion vectors stored in a cv::Mat like x0 y0 x1 y1 .. ? 2. Can I convert float* to an accepted input for cv::remap ?

Answer 1

For your #2, you can do this to get float values from pointer into some Mat:

cv::Mat testMat = cv::Mat::zeros(2, 3, CV_32F);
std::cout << "(1, 2) elem = " << testMat.at<float>(1, 2) << std::endl;

float* pTestVal = new float;
*pTestVal = 3.14F;
testMat.at<float>(1, 2) = *pTestVal;
delete pTestVal;

std::cout << "(1, 2) elem = " << testMat.at<float>(1, 2) << std::endl;

Answer 2

I've found this tutorial http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/remap/remap.html very useful for understanding how to set map1, map2 parameters. In the end it's something like this:

for(int i=0; i< rows; i++){
        for(int j=0; j<col; j++)
            { 
             map_x.at<float>(i,j) = j + motion_vect.ptr<float>(i)[2*j+0];
             map_y.at<float>(i,j) = i + motion_vect.ptr<float>(i)[2*j+1];;
        }}

To have the output image as the original :

map_x.at<float>(i,j) = j ;
map_y.at<float>(i,j) = i ;