Question

I have completely re-written this first post to better show what the problem is.

I am using ps v1.4 (highest version I have support for) and keep getting an error.
It happens any time I use any type of function such as cos, dot, distance, sqrt, normalize etc. on something that was passed into the pixelshader.
For example, I need to do "normalize(LightPosition - PixelPosition)" to use a point light in my pixelshader, but normalize gives me an error.
Some things to note-
I can use things like pow, abs, and radians with no error.
There is only an error if it is done on something passed from the vertex shader. (For example I could take the sqrt of a local pixelshader variable with no error)
I get the error from doing a function on ANY variable passed in, even text coords, color, etc.
Inside the vertex shader I can do all of these functions on any variables passed in with no errors, it's only in the pixelshader that I get an error
All the values passing from the vertex to pixel shader are correct, because if I use software processing rather than hardware I get no error and a perfectly lit scene.
Since normalizing the vector is essentially where my error comes form I tried creating my own normalizing function.
I call Norm(LightPosition - PixelPosition) and "Norm" looks like this -

float3 Norm(float3 v)
{
    return v / sqrt(dot(v, v));
}

I still get the error because I guess technically I'm still trying to take a sqrt inside the pixelshader.

The error isn't anything specific, it just says "error in application" on the line where I load my .fx file in C#
I'm thinking it could actually be a compiling error because I have to use such old versions (vs 1.1 and ps 1.4)
When debugged using fxc.exe it tells me "can not map instruction to pixel shader instruction set"

Was it helpful?

Solution

Old GPU:s didn't always support any instruction, especially in the pixel shader.

You might get away with a sqrt in the vertex shader but for a so old version (1.1 !!) the fragment shader might be extremely limited.

I.e this might not be a bug.

The work around could be to skip the hlsl and write your own assembler (but you might stumble onto the same problem there) and simulate the sqrt (say with a texture lookup and / or interpolations if you can have 2 textures in 1.0 :-p )

You can of course try to write a sqrt-lookup/interpolation in hlsl but it might be too big too (I don't remember but IIRC 1.1 don't let you write very long shaders).

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top