glreadpixels stencil buffer always throws GL_INVALID_OPERATION

Question

I'm trying to figure out stencils. Right now I am just drawing some boxes with stencil values, then reading the value. Every time I call glReadPixels with GL_STENCIL_INDEX, I get GL_INVALID_OPERATION. Here is the code in question:

glPixelStorei(GL_PACK_ALIGNMENT, 1);
GLfloat tempStencilVal = 3;
glGetError();
glReadPixels(10, g_window1Height-10, 1, 1, GL_STENCIL_INDEX, GL_FLOAT, &tempStencilVal);
if (glGetError() == GL_INVALID_OPERATION) {std::cout << "GL Invalid Operation\n";}
else {std::cout << "X: " << 10 << " Y: " << 10 << " S: " << tempStencilVal << "\n";}

I've tried 5 different data formats, 3 different glPixelStore modes, and gone over the list of glReadPixels Errors 7 times. (Yes, OGL 2.1) If I change STENCIL_INDEX to DEPTH_COMPONENT it works fine. The only thing I can't confirm is if I have a stencil buffer. Is there some initialization I'm missing or some glGet to check that?

Potentially relevant info: Win7 x64 SP1 | ASUS GTX650Ti | VS2012 Ultimate

Here is the code for the function to draw the boxes, in case that's causing it:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, g_window1Width, -g_window1Height, 0, 0.0, 50.0);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glScaled(1.0, -1.0, -1.0);
glTranslated(0.0, 0.0, 0.5);

glClearColor(0.0, 0.0, 0.0, 0.0);
glClearStencil(0);

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

glDisable(GL_DEPTH_TEST);
glEnable(GL_STENCIL_TEST);
glStencilOp( GL_REPLACE, GL_REPLACE, GL_REPLACE );

glColor3ub(0, 100, 250);

glStencilFunc(GL_ALWAYS, 1, 1);
glBegin(GL_QUADS);
glVertex3d(0, 0, 0);
glVertex3d(0, 50, 0);
glVertex3d(50, 50, 0);
glVertex3d(50, 0, 0);
glEnd();

glStencilFunc(GL_ALWAYS, 1, 1);
glBegin(GL_QUADS);
glVertex3d(g_window1Width-50, 0, 0);
glVertex3d(g_window1Width, 0, 0);
glVertex3d(g_window1Width, 50, 0);
glVertex3d(g_window1Width-50, 50, 0);
glEnd();

This isn't the first time OGL has done the wrong thing for no apparent reason, but this breaks my plan for coding the interface.

Answer 1

To check if you do have a stencil buffer, you could try doing something with the values such as drawing another quad with glStencilFunc(GL_NOTEQUAL, 1, 1); with and without the stencil test enabled.

To find the actual format used, as you say with a glGet..., it looks like glGetFramebufferAttachmentParameter will give you the answer (with the default framebuffer bound).

The stencil buffer is 8 bits (I don't think it can be anything else) so maybe change the format to GL_UNSIGNED_BYTE.

It's also possible to mix depth and stencil buffers with GL_DEPTH_STENCIL, for which you might use GL_UNSIGNED_INT_24_8. I don't what format the default framebuffer has or if this will work in your case. If you're using a library such as glut, SDL or glfw, that's responsible for setting this up and where you should look for configuring the default framebuffer.