Issue using glReadPixels result

Question

For a homework assignment, I need to implement a floodfill algorithm. My code is already structured, but I can't seem to figure out why my glReadPixels section isn't working. I was hoping someone could take a look at what I'm doing and let me know if I'm incorrect in its usage.

// I fetch pixels like so:
    glReadPixels(100, 250, 150, 150, GL_RGB, GL_UNSIGNED_BYTE, clip_pixels);
// I index the result using:
    in_rect_pos = (in_rect_y * in_rect_w) + in_rect_x;
    // in_rect_* is a point w/ its origin at (100, 250)
    GLubyte *pixel_at_pos = clip_pixels + (in_rect_pos * GL_PACK_ALIGNMENT);

But I'm not getting the correct values, and I really don't even know how to debug this.

Answer 1

I can spot one very significant mistake in your code:

GLubyte *pixel_at_pos = clip_pixels + (in_rect_pos * GL_PACK_ALIGNMENT);

Using GL_PACK_ALIGNMENT here is wrong in two different ways:

1. Fundamentally, GL_PACK_ALIGNMENT is an enum value in OpenGL. It happens to be defined to 0x0D05 (3333 in decimal), and it is not the value of the GL_PACK_ALIGNMENT GL state, which you can set via glPixelStore and query back via glGet. In both cases, GL_PACK_ALIGNMENT is just used to reference which GL state variable is to be accessed.

2. You do not use the alignment correctly. Even if you had used the value queried by glGetIntegerv(GL_PACK_ALIGNMENT, &alignment), your formula would still be wrong. The GL_PIXEL_PACK alignment reference the adresses of each row of pixel data. By default, it is set to 4, so if your row would not start at an address divisible by 4, you have to add 1 to 3 padding bytes to make it divisible by 4.

Answer 2

const int X = 0;
const int Y = 0;
const int Width = 100;
const int Height = 100;

unsigned char Buffer[Width * Height * 3];
glPixelStorei(GL_PACK_ALIGNMENT, 1);
glReadPixels(X, Y, Width, Height, GL_RGB, GL_UNSIGNED_BYTE, &Buffer[0]);

for (int i = 0; i < height; ++i)
{
    for (int j = 0; j < width; ++j)
    {
        unsigned char R = Buffer[(i * j + width) + 0];
        unsigned char G = Buffer[(i * j + width) + 1];
        unsigned char B = Buffer[(i * j + width) + 2];
    }
}

If it is aligned on some boundary (and you don't do: glPixelStorei(GL_PACK_ALIGNMENT, 1)), you will need to account for padding..

Example:

unsigned char* ptr = &Buffer[0];

for (int i = 0; i < Height; ++i)
{
    for (int j = 0; j < Width; ++j)
    {
        int R = *(ptr++);
        int G = *(ptr++);
        int B = *(ptr++);
    }

    ptr += (4 - ((width * 3) % 4)) % 4; //DWORD boundary alignment.
}