Converting cv::Mat to SDL_Texture

Question

I am trying to play a video with SDL. For that I'm using opencv to load the video, and get the frames. Then I only need to convert those frames as I need them to a SDL_Texture* and I'm ready to draw them on the screen.

That's my problem, I'm converting it to a SDL_Surface* but then the conversion to SDL_Texture is failing and I'm not sure why. Here is my code:

void Cutscene::play()
{
  this->onLoop();
  this->onRender();

  while(!frameMat.empty())
  {
    this->onLoop();
    this->onRender();
  }
}

void Cutscene::onLoop()
{
  video >> frameMat;

  convertCV_MatToSDL_Texture();
}

void Cutscene::onRender()
{
  Image::onDraw(GameEngine::getInstance()->getRenderer(), frameTexture);

}

void Cutscene::convertCV_MatToSDL_Texture()
{
  IplImage opencvimg2 = (IplImage)frameMat;
  IplImage* opencvimg = &opencvimg2;

  //Convert to SDL_Surface
  frameSurface = SDL_CreateRGBSurfaceFrom((void*)opencvimg->imageData,
                         opencvimg->width, opencvimg->height,
                         opencvimg->depth*opencvimg->nChannels,
                         opencvimg->widthStep,
                         0xff0000, 0x00ff00, 0x0000ff, 0);

  if(frameSurface == NULL)
  {
    SDL_Log("Couldn't convert Mat to Surface.");
    return;
  }

  //Convert to SDL_Texture
  frameTexture = SDL_CreateTextureFromSurface(
                    GameEngine::getInstance()->getRenderer(), frameSurface);
  if(frameTexture == NULL)
  {
    SDL_Log("Couldn't convert Mat(converted to surface) to Texture."); //<- ERROR!!
    return;
  }

  //cvReleaseImage(&opencvimg);
  //MEMORY LEAK?? opencvimg opencvimg2
}

I've used this function SDL_CreateTextureFromSurface in other parts of my project and it works there. So the question is: Do you know what is the problem with the conversion I do in my code? If not, is there a better way to do what I'm trying to do?

Answer 1

I got it to work! I think the only problem was that i had to use &frameMat and not frameMat. Here is my code if someone might be interested:

SDL_Texture* Cutscene::convertCV_MatToSDL_Texture(const cv::Mat &matImg)
{
    IplImage opencvimg2 = (IplImage)matImg;
    IplImage* opencvimg = &opencvimg2;

     //Convert to SDL_Surface
    frameSurface = SDL_CreateRGBSurfaceFrom(
                         (void*)opencvimg->imageData,
                         opencvimg->width, opencvimg->height,
                         opencvimg->depth*opencvimg->nChannels,
                         opencvimg->widthStep,
                         0xff0000, 0x00ff00, 0x0000ff, 0);

    if(frameSurface == NULL)
    {
        SDL_Log("Couldn't convert Mat to Surface.");
        return NULL;
    }

    //Convert to SDL_Texture
    frameTexture = SDL_CreateTextureFromSurface(
                    GameEngine::getInstance()->getRenderer(), frameSurface);
    if(frameTexture == NULL)
    {
        SDL_Log("Couldn't convert Mat(converted to surface) to Texture.");
        return NULL;
    }
    else
    {
        SDL_Log("SUCCESS conversion");
        return frameTexture;
    }

    cvReleaseImage(&opencvimg);

}

Answer 2

Here is another way without IplImage:

cv::Mat m ...;
// I'm using SDL_TEXTUREACCESS_STREAMING because it's for a video player, you should
// pick whatever suits you most: https://wiki.libsdl.org/SDL_TextureAccess
// remember to pick the right SDL_PIXELFORMAT_* !
SDL_Texture* tex = SDL_CreateTexture(
        ren, SDL_PIXELFORMAT_BGR24, SDL_TEXTUREACCESS_STREAMING, m.cols,
        m.rows);
SDL_UpdateTexture(tex, NULL, (void*)m.data, m.step1());

// do stuff with texture
SDL_RenderClear(...);
SDL_RenderCopy(...);
SDL_RenderPresent(...);
// cleanup (only after you're done displaying. you can repeatedly call UpdateTexture without destroying it)
SDL_DestroyTexture(tex)

I prefer this to the create surface methods because you don't need to free the surface and it is more flexible (you can update the texture easily for example, instead of create/destroy). I will also note that I could not combine these approaches: ie create the texture with SDL_CreateRGBSurfaceFrom and then later update it. That resulted in gray stripes and the image being messed up.