Normal Mapping on procedural sphere

Question

I am a student in video games, and we are working on a raytracer in C++. We are using our teachers' library.

We create procedural objects (in our case a sphere), the Camera sends a ray for each pixel of the screen and the ray send back information on what it hit.

Some of us decided to integrate Normal Maps. So, at first, we sent ray on the object, looked at the value of the Normal map texel where we hit the sphere, converted it in a vector, normalized it and sent it back in place of the normal of the object. The result was pretty good, but of course, it didn't take the orientation of the "face" (it's procedural, so there is no face, but it gives the idea) into account anymore, so the render was flat.

We still don't really know how to "blend" the normal of the texture (in tangent space) and the normal of the object together. Here is our code:

// TGfxVec3 is part of our teachers library, and is a 3d vector like this:
//    TGfxVec3( 12.7f, -13.4f, 52.0f )

// The sphere being at the origin and of radius 1, and tHit.m_tPosition being the
// exact position at the surface of the sphere where the ray hit, the normal of this
// point is the position hit by the ray.
TGfxVec3 tNormal = tHit.m_tPosition;
TGfxVec3 tTangent = Vec3CrossProduct( tNormal , m_tAxisZ );

TGfxVec3 tBiNormal = Vec3CrossProduct( tNormal , tTangent );

TGfxVec3 tTextureNorm = 2*(TGfxVec3( pNorm[0], pNorm[1], pNorm[2] )/255)-TGfxVec3( -1.0f, -1.0f, -1.0f );
// pNorm[0], pNorm[1], pNorm[2] are respectively the channels Red, Green,
// and Blue of the Normal Map texture.
// We put them in a 3D vector, divid them by 255 so their value go from 0 to 1,
// multiply them by 2, and then substract a vector, so their rang goes from -1 to +1.

tHit.m_tNorm = TGfxVec3( tTangente.x*tTextNorm.x + tCoTangente.x*tTextNorm.x + 
tNorm.x*tTextNorm.x, tTangente.y*tTextNorm.y + tCoTangente.y*tTextNorm.y + 
tNorm.y*tTextNorm.y, tTangente.z*tTextNorm.z + tCoTangente.z*tTextNorm.z + 
tNorm.z*tTextNorm.z ).Normalize();
// Here, after some research, I came across this : http://www.txutxi.com/?p=316 ,
// that allow us to convert the normal map tangent space to the object space.

The results are still not good. My main concern are the Tangent and Binormals. The Axis taken in reference (here: m_tAxisZ, the Z Axis of the Sphere), is not right. But I don't know what to take, or even if what I am doing is really good. So I came here for help.

Answer 1

You are mostly right and completely wrong at the same time.

Tangent space normal mapping use a transformation matrix to convert the tangent space normal from the texture to another space, like object or world space, or transform the light in the tangent space to compute the lighting with everything in the same space.

Bi-normal is a common mistake and should be named bi-tangent.

It is sometime possible to compute the TBN at the fly on simple geometry, like on a height-map as it is easy to deduce the tangent and the bi-tangent on a regular grid. But on a sphere, the cross trick with a fixed axis will result to a singularity at the pole where the cross product give a zero length vector.

Last, even if we ignore the pole singularity, the TBN must be normalized before you apply the matrix to the tangent space normal. You may also miss a transpose, as a 3x3 orthonormal matrix inverse is the transpose, and what you need is the inverse of the original TBN matrix if you go from tangent to object.

Because of all this, we most often store the TBN as extra information in the geometry, computed from the texture coordinate ( the url you referenced link to that computation description ) and interpolate at runtime with the other values.

Rem : there is a rough simplification to use the geometry nornal as the TBN normal but there is no reason in the first place that they match.

Answer 2

So, we finally did it. :D Ok, I will try to be clear. For this, two images :

(1) : http://i.imgur.com/cHwrR9A.png

(2) : http://i.imgur.com/mGPH1RW.png

(My drawing skill has no equal, I know).

So, the main problem was to find the Tangent "T" and the Bi-tangent "B". We already have the Normal "N". Our circle always being at the origin with a radius of 1, a point on its surface is equal to the Normal to that point (black and red vector on the first image). So, we have to find the tangent to that point (in green). For this, we just have to rotate the vector from PI/2 rad :

With N( x, y ) :

T = ( -N.y , N.x )

However, we are in 3D. So the point will not always be at the equator. We can easily solve this problem by ignoring the position in Y of our point and normalize the vector with only the two other component. So, on the second image, we have P (we set its Y value to 0), and we normalize the new vector to get P'.

With P( x, y, z ) :

P' = ( P.x, 0, P.z).Normalize();

Then, we go back to my first message to find the T. Finally, we get the B with a cross product between the N en the T. Finally, we calculate the normal to that point by taking the normal map into account.

With the variable "Map" containing the three channels (RGB) of the normal Map, each one clamped from -1 to 1, and T, N and B all being 3D vectors :

( Map.R*T + Map.G*B + Map.B*N ).Normalize();

And that's it, you have the normal to the point taking your normal map into account. :) Hope this will be usefull for others.