BasicEffect fog, code used

Question

I want to copy BasicEffect's fog method to use in my own shader so I don't have to declare a basiceffect shader and my own. The HLSL code of the basic effect was released with one of the downloadable samples on XNA Creators Club a while ago and I thought the method needed would be found within that HLSL file. However, all I can see is a function being called but no actual definition for that function. The function called is:

ApplyFog(color, pin.PositionWS.w);

Does anybody know where the definition is and if it's freely acceptable. Otherwise any help on how to replicate it's effect would be great.

I downloaded the sample from here.

Thanks.

Edit: Stil having problems. Think it's to do with getting depth:

VertexToPixel InstancedCelShadeVSNmVc(VSInputNmVc VSInput, in VSInstanceVc VSInstance)
{   
    VertexToPixel Output = (VertexToPixel)0;
    Output.Position = mul(mul(mul(mul(VSInput.Position, transpose(VSInstance.World)), xWorld), xView), xProjection);
    Output.ViewSpaceZ = -VSInput.Position.z / xCameraClipFar;

Is that right? Camera clip far is passed in as a constant.

Answer 1

Heres an example of how to achieve a similar effect

In your Vertex Shader Function, you pass the viewspace Z position, divided by the distance of your farplane, that gives you a nice 0..1 mapping for your depthvalues.

Than, in your pixelshader, you use the lerp function to blend between your original color value, and the fogcolor, heres some (pseudo)code:

cbuffer Input //Im used to DX10+ remove the cbuffer for DX9
{
    float FarPlane;
    float4 FogColor;
}

struct VS_Output
{
    //...Whatever else you need
    float ViewSpaceZ : TEXCOORD0; //or whatever semantic you'd like to use
}


VS_Output VertexShader(/*Your Input Here */)
{
    VS_Output output;
    //...Transform to viewspace
    VS_Output.ViewSpaceZ = -vsPosition.Z / FarPlane;

    return output;
}

float4 PixelShader(VS_Output input) : SV_Target0 // or COLOR0 depending on DX version
{
    const float FOG_MIN = 0.9;
    const float FOG_MAX = 0.99;
    //...Calculate Color
    return lerp(yourCalculatedColor, FogColor, lerp(FOG_MIN, FOG_MAX, input.ViewSpaceZ));
}

I've written this from the top of my head, hope it helps. The constants i've chose will give you a pretty "steep" fog, choose a smaller value for FOG_MIN to get a smoother fog.