Relation between pixel formats of ofGrabber and ofTexture

Question

I'm currently creating an iOS app and I'm having trouble understanding the relationship between taking pixels from an ofGrabber and drawing them using an ofTexture.

My current code:

In setup():

//Set iOS orientation
ofSetOrientation(OF_ORIENTATION_90_LEFT);

//Inits the camera to specified dimensions and sets up texture to display on screen
grabber.initGrabber(640, 480, OF_PIXELS_BGRA); //Options: 1280x720, 640x480

//Allocate opengl texture
tex.allocate(grabber.width, grabber.height, GL_RGB);

//Create pix array large enough to store an rgb value for each pixel
//pix is a global that I use to do pixel manipulation before drawing
pix = new unsigned char[grabber.width * grabber.height * 3];

In update()

//Loads the new pixels into the opengl texture
tex.loadData(pix, grabber.width, grabber.height, GL_RGB);

In draw():

CGRect screenBounds = [[UIScreen mainScreen] bounds];
CGSizeMake screenSize = CGSizeMake(screenBounds.size.width, screenBounds.size.height);

//Draws the texture we generated onto the screen
//On 1st generation iPad mini: width = 1024, height = 768
tex.draw(0, 0, screenSize.height, screenSize.width); //Reversed for 90 degree rotation

What I'm wondering:

1) Why does the ofGrabber and the ofTexture use seemingly different pixel formats? (These formats are the same used in the VideoGrabberExample)

2) What exactly is the texture drawing with the resolution? I'm loading the pix array into the texture. The pix array represents a 640x480 image, while the ofTexture is drawing a 1024x768 (768x1024 when rotated) image to the screen. How is it doing this? Does it just scale everything up since the aspect ratio is basically the same?

3) Is there a list anywhere that describes the OpenGL and OpenFrameworks pixel formats? I've searched for this but haven't found much. For example, why is it OF_PIXELS_BGRA instead of OF_PIXELS_RGBA? For that matter, why does my code even work if I'm capturing BGRA formatted data (which I assume included a gamma value) yet I am only drawing RGB (and you can see that my pix array is sized for RGB data).

I might also mention that in main() I have:

ofSetupOpenGL(screenSize.height, screenSize.width, OF_FULLSCREEN);

However, changing the width/height values above seem to have no effect whatsoever on my app.

Answer 1

1. ofGrabber is CPU based, so it uses OF_PIXELS_BGRA by the programmers choice. It is common for some cameras to have BGRA pixel order, so this just avoids the grabber to perform a costly memcpy when grabbing from the source. ofTexture maps GPU memory, so it maps to what you'll see on screen (RGB). Note that GL_RGB is an OpenGL definition.

2. ofTexture does scale to whatever you tell it to. This is done in GPU so it's quite cheap. It does not need to have the same aspect ratio.

3. This is quite up to the programmer or your requirements. Some cameras provide BGRA streams, other cameras or files provide RGB directly, or even YUV-I420. Color formats are very heterogeneous. OpenFrameworks will handle conversions in most cases, look into ofPixels to see where and how it's used. In a nutshell:

   • OF_PIXELS_XXX : Used by ofPixels, basically a RAM mapped bitmap
   • OF_IMAGE_XXX : Used by ofImage, which wrapps ofPixels and makes it simpler to use
   • GL_XXX : Used by OpenGL and ofTexture, low level GPU Memory mapped