Normal map from height map artifact

Question

I'm baking normal map on the fragment shader from a height map. The height map looks great and looks smooth. However, when I generate the normal map I get very weird result.

Here are two rendered images that show the problem, one with all lighting calculations and second one has the normal map image applied on top of the mesh.

The way I bake the normal map is by sampling neighbor pixels on the fragment shader.

The mesh is 32x32 and the normal map and height map is 64x64. Here's the fragment shader code that samples the neighbor pixels:

float NORMAL_OFF = (1.0 / 64.0);
vec3 off = vec3(-NORMAL_OFF, 0, NORMAL_OFF);

// s11 = Current
float s11 = texture2D(uSampler, texturePos).x;

// s01 = Left
float s01 = texture2D(uSampler, vec2(texturePos.xy + off.xy)).x;

// s21 = Right
float s21 = texture2D(uSampler, vec2(texturePos.xy + off.zy)).x;

// s10 = Below
float s10 = texture2D(uSampler, vec2(texturePos.xy + off.yx)).x;

// s12 = Above
float s12 = texture2D(uSampler, vec2(texturePos.xy + off.yz)).x;

vec3 va = normalize( vec3(off.z, 0.0, s21 - s11) );
vec3 vb = normalize( vec3(0.0, off.z, s12 - s11) );

vec3 normal = normalize( cross(va, vb) );

texturePos is calculated on the vertex shader as vertexPosition.x / 128 (128 because the distance between vertices is 4 pixels. So 32 * 4 = 128).

Why is my result so weird?

Answer 1

Your height map has too few sampling depth resolution, resulting in those hard steps. Probably your height map is 8 bit, giving you a max of 256 height levels. Now if your height map is is higher planar resolution than 256, the lateral resolution is insufficient to represent a smooth heightfield.

Solution: Use a higher sampling resolution for your height map. 16 bits are a popular choice.

Your shader and baking code are fine though, they just don't get input data with enough resolution to work with.