OpenCV: Merge separated JPEG bayer channels

Question

I have a camera that is giving 4 separated JPEG images for the 4 different Bayer channels (B,G1,G2,R).

I want to transform this in to a colour image.

What I'm doing at the moment is uncompress the jpeg, restore the "original" image manually and converting to a colour image using cvtColor. But this is too slow. How could I do it better?

    cv::Mat imgMat[4]=cv::Mat::zeros(616, 808, CV_8U); //height, width
    for (k=0;k<4;k++) {
        ........
        imgMat[k] = cv::imdecode(buffer, CV_LOAD_IMAGE_GRAYSCALE);
    }
    //Reconstruct the original image from the four channels! RGGB
    cv::Mat Reconstructed=cv::Mat::zeros(1232, 1616, CV_8U);
    int x,y;
    for(x=0;x<1616;x++){
        for(y=0;y<1232;y++){
            if(y%2==0){
                if(x%2==0){
                    //R
                    Reconstructed.at<uint8_t>(y,x)=imgMat[0].at<uint8_t>(y/2,x/2);
                }
                else{
                    //G1
                    Reconstructed.at<uint8_t>(y,x)=imgMat[1].at<uint8_t>(y/2,floor(x/2));
                }
            }
            else{
                if(x%2==0){
                    //G2
                    Reconstructed.at<uint8_t>(y,x)=imgMat[2].at<uint8_t>(floor(y/2),x/2);
                }
                else{
                    //B
                    Reconstructed.at<uint8_t>(y,x)=imgMat[3].at<uint8_t>(floor(y/2),floor(x/2));
                }
            }
        }
    }
    //Debayer
    cv::Mat ReconstructedColor;
    cv::cvtColor(Reconstructed, ReconstructedColor, CV_BayerBG2BGR);

It seems clear that what it takes more time is decoding the jpeg images. Has somebody some advice/trick I could use to speed up this code?

Answer 1

Firstly you should do a profile to see where the time is mostly going. Maybe it is all in imdecode(), as "seems clear", but you might be wrong.

If not, .at<>() is a bit slow (and you are calling it nearly 4 million times). You can get some speedup by more efficent scanning of the image. Also you do not need floor() - that will avoid converting an int to double and back again (2 million times). Something like this will be faster:

int x , y;
for(y = 0; y < 1232; y++){
    uint8_t* row = Reconstructed.ptr<uint8_t>(y);
    if(y % 2 == 0){
        uint8_t* i0 = imgMat[0].ptr<uint8_t>(y / 2);
        uint8_t* i1 = imgMat[1].ptr<uint8_t>(y / 2);

        for(x = 0; x < 1616; ){
            //R
            row[x] = i0[x / 2];
            x++;

            //G1
            row[x] = i1[x / 2];
            x++;
        }
    }
    else {
        uint8_t* i2 = imgMat[2].ptr<uint8_t>(y / 2);
        uint8_t* i3 = imgMat[3].ptr<uint8_t>(y / 2);

        for(x = 0; x < 1616; ){
            //G2
            row[x] = i2[x / 2];
            x++;

            //B
            row[x] = i3[x / 2];
            x++;
        }
    }
}