Using glReadPixels to read value at Tap Point ? OpenGL 2.0 iOS

Question

Say I have a square which is defined as follows :

typedef struct {
float Position[3];
float Color[4];
} Vertex;

const Vertex Vertices[] = {
{{2, 0, 0}, {1, 0, 0, 1}},
{{4, 0, 0}, {1, 0, 0, 1}},
{{4, 2, 0}, {1, 0, 0, 1}},
{{2, 2, 0}, {1, 0, 0, 1}}
};

const GLubyte Indices[] = {
0, 1, 2,
2, 3, 0
};

And I am applying the following projection and modelview matrices :

- (void)update {

//Projection matrix.
float aspect = fabsf(self.view.bounds.size.width / self.view.bounds.size.height);
projectionMatrix = GLKMatrix4MakePerspective(GLKMathDegreesToRadians(_projectionAngle), aspect, 4.0f, 10.f);
self.effect.transform.projectionMatrix = projectionMatrix;

//Modelview matrix.
modelViewMatrix = GLKMatrix4MakeTranslation(_xTranslation, _yTranslation, -7.0);
self.effect.transform.modelviewMatrix = modelViewMatrix;

}

I now want to read the pixel colour of the object where a user taps on the screen. I am trying glkMathUnproject in combination with glReadPixels as follows, however glReadPixels is returning incorrect colour values for the tap point :

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
UITouch *touch = [touches anyObject];
CGPoint tapLoc = [touch locationInView:self.view];

bool testResult;

GLint viewport[4];
glGetIntegerv(GL_VIEWPORT, viewport);

GLKVector3 nearPt = GLKMathUnproject(GLKVector3Make(tapLoc.x, (tapLoc.y-1024)*-1, 0.0), modelViewMatrix, projectionMatrix, &viewport[0] , &testResult);

GLKVector3 farPt = GLKMathUnproject(GLKVector3Make(tapLoc.x, (tapLoc.y-1024)*-1, 1.0), modelViewMatrix, projectionMatrix, &viewport[0] , &testResult);

farPt = GLKVector3Subtract(farPt, nearPt);

GLubyte pixelColor[4];
glReadPixels(farPt.v[0], farPt.v[1], 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, &pixelColor[0]);
NSLog(@"pixelColor %u %u %u %u", pixelColor[0],pixelColor[1],pixelColor[2], pixelColor[3]);

}

Can anyone advise how I can get the pixel colour accurately ?

Answer 1

Guy Cogus' code looks like the right approach, but you should know why what you posted isn't working. The point of unprojecting the vector using GLKUnproject is to get it out of screen space and into object space (the coordinate system of your geometry before it is transformed by the projection or modelview matrices), not into screen space, which is what glReadPixels is in (by definition, since you are reading pixels).

Answer 2

Check out cocos2d and see how they're doing the following inspection.

Here's some code that I'm using to get the tapped pixel colour. Hope it helps.

I've actually changed the code a bit to run on a whole scene, but you can do the same on a specific node. If you do, make sure to translate it to {0,0} with the anchor-point at {0,0} and the identity transform so that it displays properly, then reset it when you're done.

    CCScene *runningScene = [CCDirector sharedDirector].runningScene;
    CGRect boundingBox = runningScene.boundingBox;
    CCRenderTexture *renderTexture = [[CCRenderTexture alloc] initWithWidth:(int)CGRectGetWidth(boundingBox)
                                                                     height:(int)CGRectGetHeight(boundingBox)
                                                                pixelFormat:kCCTexture2DPixelFormat_RGBA8888];
    [renderTexture begin];

    [runningScene visit];

    // Get the colour of the pixel at the touched point
    CGPoint location = ccp((point.x - CGRectGetMinX(boundingBox)) * CC_CONTENT_SCALE_FACTOR(),
                           (point.y - CGRectGetMinY(boundingBox)) * CC_CONTENT_SCALE_FACTOR());
    UInt8 data[4];
    glReadPixels((GLint)location.x,(GLint)location.y, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, data);

    [renderTexture end];

    // data[0] = R
    // data[1] = G
    // data[2] = B
    // data[3] = A