gluUnProject returns an inaccurate z coordinate under some conditions

Question

I am getting some troubles with gluUnProject because I am retrieving an inaccurate Z coordinate.

The program involved is essentially a 3d viewer. I implemented a little function the allows the user to click on the loaded model and retrieve the distance between two points.

This is the code:

//  Declare variables for viewport, modelview and projection matrix
int viewport[] = new int[4];
float modelview[] = new float[16];
float projection[] = new float[16];
//  GL y-coordinate position
int realY;
//  Returned [wx, wy, wz, 1] coordinates
float worldCoordinates[] = new float[4];

gl2.glGetIntegerv(GL2.GL_VIEWPORT, viewport, 0);
gl2.glGetFloatv(GL2.GL_MODELVIEW_MATRIX, modelview, 0);
gl2.glGetFloatv(GL2.GL_PROJECTION_MATRIX, projection, 0);

//  Retrive the Y coordinate, note viewport[3] is height of window in pixels
realY = viewport[3] - click.y - 1;

//  Allocate a buffer to store the result
FloatBuffer fb = FloatBuffer.allocate(1);

//  Retrieve the Z coordinate, read a block of pixels from the frame buffer
gl2.glReadPixels(click.x, realY, 1, 1, GL2.GL_DEPTH_COMPONENT, GL2.GL_FLOAT, fb);

System.out.println("Depth: "+fb.get(0));

//  Avoid centering if the click is on nothing
if (fb.get(0) == 1.0f) {
    return null;
}
//  Map window coordinates to object coordinates, retrieve the world coordinates
glu.gluUnProject(click.x, realY, fb.get(0), modelview, 0, projection, 0,
            viewport, 0, worldCoordinates, 0);

When I am measuring the distance between two points lying on a surface perpendicular to the user everything works properly. If we image a 3-axis system, with z pointing toward the user and y toward the top, the two points lying on the XY plane. It works regardless whatever rotation around Z.

For example, here I created a small square plane and both two points have the same height (one is obscured by the label):

World coordinates P1: (0.019062456, 0.03357359, 0.08333321, 1)

World coordinates P2: (0.025983132, 0.028048635, 0.08333321, 1)

But if I rotate the entire scene over the global X axes, then the retrieved Z coordinates change:

X and Y coordinates don't matter, what matters is the Z one. It is no more 0.08333321.

World coordinates P1: (0.026863147, 0.027185505, 0.0831252, 1)

World coordinates P2: (0.020072304, 0.034560144, 0.08312501, 1)

Moreover after a rotation over global X also the zooming affects the Z coordinate of the points.

Ps: I am using JOGL. Zooming in and out is performed by enlarging/reducing the glOrtho parallelepiped. The Z depth produced by glReadPixels, fb.get(0), is included in [0, 1].

Where am I wrong?

Answer 1

Are you expecting to see the exact coordinate of the point on your plane? You're not going to get it this way.

When you render your square, the coordinates are transformed into 3D, and the primitive rasterised (i.e. converted to 2D and interpolated), generating screen X,Y and a (typically) 24-bit depth value. That's quite a loss of precision (and it's not even linear), so if you read it and invert the transform, you're getting only an approximation to the original value.

If your polygon happens to be perpendicular to the view direction, the interpolation will at least give a consistent z-value across the surface (as all transformed Z-values are the same), though it's not going to be necessarily the right z-value. So any change to the depth will result in the error changing, and if your plane is not perpendicular it stops even being consistent across the poly.

Depth is basically good enough for drawing without too many errors (and even then, not always), but not for reconstructing accurate geometry.

If you want better results, you could rasterise the actual model-space X,Y,Z to a (float) FBO with a shader, and read that back. Alternatively, intersect a ray with your geometry in software.