How to check if an object lies outside the clipping volume in OpenGL?

Question

I'm really confused about OpenGL's modelview transformation. I understand all the transformation processes, but when it comes to projection matrix, I'm lost :(

If I have a point P (x, y, z), how can I check to see if this point will be drawn on a clipping volume defined by either by parallel clipping volume or perspective clipping volume? What's the mathematical background behind this process?

Answer 1

Apply the model-view-projection matrix to the object, then check if it lies outside the clip coordinate frustum, which is defined by the planes:

    -w < x < w
    -w < y < w
     0 < z < w

So if you have a point p which is a vec3, and a model-view-projection matrix, M, then in GLSL it would look like this:

    bool in_frustum(mat4 M, vec3 p) {
        vec4 Pclip = M * vec4(p, 1.);
        return abs(Pclip.x) < Pclip.w && 
               abs(Pclip.y) < Pclip.w && 
               0 < Pclip.z && 
               Pclip.z < Pclip.w;
    }

Answer 2

To determine if a given point will be visible on the screen, you test it against the viewing frustum. See this frustum culling tutorial:

http://www.lighthouse3d.com/tutorials/view-frustum-culling/