Bad results after transforming screen to 3D world coordinates with GLM + Answer

Question

I'm working on a 3D android game and I'm new in it. I'm using GLM, Android NDK, Opengl ES 2.0.

When I touch the screen, I would like to get that 3D point in world space.

Parts of my code:

//Camera data
float eye[3]  = {0.0f, 11.0f, -2.0f};
float look[3] = {0.0f, -1.0f, 3.0f};
const float up[3] = {0.0f, 1.0f, 0.0f};

glViewport(0, 0, width, height);
...
const float ratio = (float) width / height;
mViewMatrix = glm::lookAt(glm::make_vec3(eye), glm::make_vec3(look), glm::make_vec3(up));
mProjMatrix = glm::perspective(30.0f, -ratio, near, far); // near = 1.0f; far = 100.0f;
mMVPMatrix = mProjMatrix * mViewMatrix;
....

I tried with this:

glm::vec3 worldXYZ = glm::unProject(glm::vec3(x, height - y, 1.0f), mViewMatrix, mProjMatrix, glm::vec4(0, 0, width, height));

where x and y is the screen coordinates in pixel, but I get weird results. When I touch example the origo, the result should be something like (0,0,0), but I get this: (0.5, -89.6, 16).

Where did I make mistake?

---

Update with my answer:

Andon M. Coleman was right, and I made changes in my code according to his answer. I did not use the glm::unProject method, because I ended up getting bad results, so I took a step back and I calculated things myself.

float xx = (2.0f * x) / width - 1.0f; // between [-1;+1]
float yy = (2.0f * (height - y)) / height - 1.0f;  // between [-1;+1]
float zzN = 0.0f; // near
float zzF = 1.0f; // far

glm::mat4 invM = glm::inverse(mMVPMatrix);

glm::vec4 mmN = invM * glm::vec4(xx, yy, zzN, 1.0f);
glm::vec4 mmF = invM * glm::vec4(xx, yy, zzF, 1.0f);

normalizing the vectors:

glm::vec3 nn = glm::vec3(mmN[0] / mmN[3], mmN[1] / mmN[3], mmN[2] / mmN[3]);
glm::vec3 ff = glm::vec3(mmF[0] / mmF[3], mmF[1] / mmF[3], mmF[2] / mmF[3]);

and then I needed the ray (which goes from nn to ff). I used this line equation:

x = x1+(x2-x1)*t

y = y1+(y2-y1)*t

z = z1+(z2-z1)*t

where P(x,y,z) are the points in the ray, nn=(x1,y1,z1) and ff=(x2,y2,z2) are the 2 points which define the ray. I was interested in the point where y=0 (the ray and the plane are meeting), so I ended up calculating the x and z world-coordinates like this:

float t = nn[1] / (nn[1] - ff[1]);
float wx = nn[0] + (ff[0] - nn[0]) * t;
float wz = nn[2] + (ff[2] - nn[2]) * t;

Answer 1

Window-space ("screen coordinates") Z=1.0 is your far plane.

You have solved for the most distant point that projects to (x, height - y). That is not particularly useful most of the time.

There is no way to get a single point in world-space from 2D window-space coordinates; projection does not work that way.

You should solve for the closest point as well (Z=0.0), then you can cast a ray that passes through both points. The resulting ray will represent the infinite number of points that project to (x,y,*).