eye-space pixel width in GLSL vertex shader

Question

I want to compute the eye-space width of a pixel's projected pyramid at the current vertex location in a glsl vertex shader, but I can't seem to get the math right. Here is an obviously incorrect example:

// GLSL VERTEX SHADER
#version 410 compatibility

uniform vec4 viewport; // same as glViewport​
in vec4 gl_Vertex;

void main ()
{
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
    float pixelWidth = gl_Position.z / viewport.z;
<snip>

But this does not account for the FOV or clipping planes.

Answer 1

I worked through the math and figured it out. :) As I had hoped there are no additional matrix transformations required, just one divide:

// GLSL VERTEX SHADER
#version 410 compatibility

uniform vec4 viewport; // same as glViewport​
in vec4 gl_Vertex;

float pixelWidthRatio = 2. / (viewport.z * gl_ProjectionMatrix[0][0]);

void main ()
{
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
    float pixelWidth = gl_Position.w * pixelWidthRatio;
    <snip>

Or alternatively:

<snip>
float pixelHeightRatio = 2. / (viewport.w * gl_ProjectionMatrix[1][1]);

void main ()
{
    gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
    float pixelHeight = gl_Position.w * pixelHeightRatio;
<snip>

As expected, pixelWidth and pixelHeight are the same if the pixels are square.

Answer 2

In a vertex shader there is no such thing as a world-space width of a pixel. "Width" is projected into a pixel only at the given Z-distance. In general you have a pyramid projected into a pixel.

Here you go:

1. Convert two screen-points into NDC points:

       vec2 Screen = vec2( ScreenWidth, ScreenHeight );
       vec3 Point1 = vec3( ScreenPt1 / Screen * 2.0 - vec2( 1.0 ), 1.0 );
       vec3 Point2 = vec3( ScreenPt2 / Screen * 2.0 - vec2( 1.0 ), 1.0 );
   

2. Unproject NDC points into world-space positions:

       vec4 R1 = vec4( Point1, 1.0 );
       vec4 R2 = vec4( Point2, 1.0 );
   
       R1 = Projection.GetInversed() * R1;
       R1 = ModelView.GetInversed() * R1;
       R1 /= R1.W;
   
       R2 = Projection.GetInversed() * R2;
       R2 = ModelView.GetInversed() * R2;
       R2 /= R2.W;
   

3. Find the distance between R1 and R2.

In a fragment shader you can use local derivatives. Take a look here:

http://www.opengl.org/sdk/docs/manglsl/xhtml/dFdx.xml

and here:

http://www.opengl.org/sdk/docs/manglsl/xhtml/fwidth.xml

Available only in the fragment shader, dFdx and dFdy return the partial derivative of expression p in x and y, respectively. Deviatives are calculated using local differencing. Expressions that imply higher order derivatives such as dFdx(dFdx(n)) have undefined results, as do mixed-order derivatives such as dFdx(dFdy(n)). It is assumed that the expression p is continuous and therefore, expressions evaluated via non-uniform control flow may be undefined.