Calculating Per-Vertex Normals from 3D Noise

Question

I have currently implemented a 3D noise function in GLSL which is used to displace the vertices of a sphere to give terrain. I am currently using the geometry shader to simply calculate per-face normals (I have tessellation too, hence why I am doing this here instead of the vertex shader). Now I would like to calculate per-vertex normals.

Now I have seen a few posts about calculating normals from noise when displacing a flat grid but can't seem to get it working for myself. Below is a snippet from my tessellation evaluation shader which calculates the position of the new vertex. (This works fine).

// Get the position of the newly created vert and push onto sphere's surface.
tePosition = normalize(point0 + point1 + point2) * Radius;

// Get the noise val at given location. (Using fractional brownian motion)
float noiseVal = fBM(tePosition, Octaves); 

// Push out vertex by noise amount, adjust for amplitude desired.
tePosition = tePosition + normalize(tePosition) * noiseVal * Amplitude; 

tePosition then goes into the geometry shader where three of them are used to calculate the surface normal of the triangle. How would I go about using this to then calculate the normal for said vertex?

I have attempted to do the "neighbour" method by resampling the noise at two small offsets from tePosition (I push these back onto the sphere before displacing them by the noise value). Then using these two new positions, I get the vector from tePosition to each of them and use the cross product to get a normal. However this results in many areas being black (suggesting the normal is backwards) and the parts where the normals are facing outwards seem to be quite uniform around the sphere (illumination on opposite side of light). Here's the code that does the above:

// theta used for small offset
float theta = 0.000001; 
// Calculate two new position on the sphere.
vec3 tangent = tePosition + vec3(theta, 0.0, 0.0);
tangent = normalize(tangent) * Radius;
vec3 bitangent = tePosition + vec3(0.0, theta, 0.0);
bitangent = normalize(bitangent) * Radius; 

// Displace new positions by noise, then calculate vector from tePosition
float tanNoise = fBM(tangent, Octaves) * Amplitude;
tangent += normalize(tangent) * tanNoise;
tangent = tangent - tePosition;

float bitanNoise = fBM(bitangent, Octaves) * Amplitude;
bitangent += normalize(bitangent) * bitanNoise;
bitangent = bitangent - tePosition;

vec3 norm = normalize(cross(normalize(tangent), normalize(bitangent)));

I have tried varying the theta value and changing how this is used to offset, which results in varying degrees of "wrongness".

Does anybody have any ideas as to how I can calculate the normals correctly?

Answer 1

The vectors you add to construct the tangents, (theta, 0.0, 0.0) and (0.0, theta, 0.0), are not tangent to the sphere. To get a tangent and bitangent, you should use the cross product:

// pos x (1,0,0) could be 0, so add pos x (0,1,0).
vec3 vecTangent = normalize(cross(tePosition, vec3(1.0, 0.0, 0.0))
  + cross(tePosition, vec3(0.0, 1.0, 0.0)));
// vecTangent is orthonormal to tePosition, compute bitangent
// (rotate tangent 90° around tePosition)
vec3 vecBitangent = normalize(cross(vecTangent, tePosition));

vec3 ptTangentSample = noisy(tePosition + theta * normalize(vecTangent));
vec3 ptBitangentSample = noisy(tePosition + theta * normalize(vecBitangent));

vec3 vecNorm = normalize(
  cross(ptTangentSample - tePosition, ptBitangentSample - tePosition));

As a side note, I would advise against using the same variables for vectors (direction+length, mathematically (x,y,z,0)) and points (positions in a coordinate system, mathematically (x,y,z,1)).