Java OpenGL screen to world transformation using gluUnProject + design discussion

Question

I've got 2 questions. One's a specific question about why my code isn't working as intended and the other is a design question.

1 (Specific Question): I'm trying to map screen coordinates to world coordinates in a 2D tile-based engine (uses the x/y axis, z = 0). I've used a Nehe's tutorial port on how to achieve this but the results I get aren't as expected.

I have a class called MouseController. Whenever a mouse event is trigger (via swing's MouseListener), I pass the MouseEvent into my MouseController.

In my GLCanvas's draw function, I call the MouseController.processClick(GL) function to be able to pass the current gl context into the processClick function and grab the modelview, projection matrices and viewport.

When I click on a block rendered on the screen, the world coordinates that are given back to me make little to no sense. For one, I would expect the z value to be 0, but its 9 (which is how high my camera is set to), and my x and y values are always really close to 0 (occasionally jumping up to 1-9 with very slight movements, then back to a number very close to 0).

Anyone have any idea why this might be the case? The processClick function is below:

    public void processClick(GL gl) {

        int x = e.getX();
        int y = e.getY();

        if(e.getButton() == MouseEvent.BUTTON1) {

            gl.glGetIntegerv(GL.GL_VIEWPORT, viewPort, 0);
            gl.glGetDoublev(GL.GL_MODELVIEW_MATRIX, mvMatrix, 0);
            gl.glGetDoublev(GL.GL_PROJECTION_MATRIX, prMatrix, 0);

            int realy = viewPort[3] - y;

            glu.gluUnProject((double)x, (double)realy, 0, mvMatrix, 0, prMatrix, 0, viewPort, 0, wCoord, 0);

            System.out.println(x + " " + y);
            System.out.println(x + " " + realy);
            System.out.println(wCoord[0] + " " + wCoord[1] + " " + wCoord[2]);
        }

        e = null;
    }

Sample output I get from the above function when I click on the screen where I rendered a square at world coordinates (4,5,0):

  878 56
  878 636
  0.0445182388817236 0.055475957454737095 8.900000001489369

Thanks!

EDIT: Reading in the depth buffer using glReadPixels and using that as the z (which returns 1) gets me results that are kind of right, but are too big by a factor of 20.

EDIT2: If I set the far clipping plane to the same value as the height of the camera, it seems to work (but this isn't really a fit).

2 (Design Question): I feel as if it doesn't make sense to process clicks in the OpenGL canvas' draw function. I seem to require the GL from that function to be able to retrieve the matrices necessary for gluUnProject to run. How would you guys design this? Ideally I feel as if I should be able to have this run completely separate of the draw function. Store a reference to the gl object into my MouseController and process the click when the MouseListener picks up on it.

Input about how you guys would/do handle it would be much appreciated too!

Answer 1

I decided just to use gluOrtho2D to setup my projection to simplify solving this problem.

The only "tricky" thing was passing in the actual screen resolution to gluOrtho2D, which could be retrieved (and stored) with:

getContentPane().getSize()

From the JFrame I used.

Setting the preferred size on the panel then packing the frame achieved close results, but the panel itself was still slightly off.