Float32Array not updating ArrayBuffer to reflect assigned values when interacting with WebGL (Chrome)

Question

In order to manually pack and bind an RGBA float component texture to the GPU using WebGL in chrome using the OES_texture_float extension, pixel component data must be stored in a Float32Array.

For example, for a simple 3 pixel texture, with 4 float components each, a plain JS array would first be declared:

var pixels = [1.01, 1.02, 1.03, 1.04, 2.01, 2.02, 2.03, 2.04, 3.01, 3.02, 3.03];

Then to convert the plain JS array to a strongly typed array of floats which can be fed to the GPU, we simply use the Float32Array constructor which can take a plain JS array of numbers as input:

pixels = new Float32Array(pixels);

Now that we have our texture represented as a strongly typed array of floats, we can feed it to the GPU using an already established WebGL context (which is working and beyond the scope of this question), using texImage2D:

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 3, 1, 0, gl.RGBA, gl.FLOAT, pixels);

Making the appropriate render calls shows that these floats are being passed into, and back out of (by encoding an output float into the fragment color) the GPU without error (albeit a slight loss of precision due to conversions).

The Problem

Converting from a plain JS array to a Float32Array is actually a pretty expensive operation, and it's far quicker to manipulate already converted floats in the Float32Array - an operation that seems to be supported according to most JS specs floating around: https://developer.mozilla.org/en-US/docs/Web/API/Float32Array

Once established, you can reference elements in the array using the object's methods, or using standard array index syntax (that is, using bracket notation).

The problem occurs when:

a Float32Array is created with a plain JS array of preset values

we change one or many values in the Float32Array using [] notation, ie:

pixels[0] = 420.4;
pixels[1] = 420.4;

we pass the Float32Array to the GPU using texImage2D, and using the same method mentioned above shows that the inititally set values to the Float32Array somehow made it to the GPU without the two values changed to 420.4

WTF?

My best guess is that because strongly typed arrays are (usually) internally represented as a buffer and a view, that I'm updating the view, and the buffer is not reflecting the changes. Logging the Float32Array to the browser console shows that the two changed numbers in this case appear to indeed be changed. However because the contents of an ArrayBuffer can't be read through the console in Chrome, it's a debugging dead end as far as my skillset goes.

Trying the same create, convert, update and check methodology (without the GPU involved) using a NodeJS REPL, reveals that the buffer values are updated during the evaluation of pixels[0] = 420.4; and are not updated in a 'lazy' fashion when the buffer is read.

It may be possible that Chrome is lazy updating underlying buffers, but copying that data to the GPU does not trigger the getter but rather copies it raw from memory.

Temporary Solution

Until the underlying problem is found and remedied (if even applicable), it would seem that Float32Arrays are essentially immutable (cannot be changed) once created in the context of WebGL textures. There also appears to be a .set() method attached to typed arrays, but this:

 pixels.set(new Float32Array([420.4]), index);

Seems like a lot of external boxing/conversion to get around a lazy buffer, especially one that claims to allow [] access.

Answer 1

Typed Arrays including Float32Arrays are just packed arrays (think C/C++). Updating them is instant. If you want to see data on the GPU you have to upload the data again with texImage2D but otherwise, there's no magic, no crazy buffering, very straight forward. If you know C/C++ it's functionally equivalent to

void* arrayBuffer = malloc(sizeOfBuffer);
float* viewAsFloat = (float*)arrayBuffer;

Typed Arrays are not views into JS arrays. Using a native JS array to initialize a typed array is just a convenient way of initializing the typed array. Once created the TypedArray is a new array.

You can get multiple ArrayBuffer views into the same ArrayBuffer though.

Example

var b = new ArrayBuffer(16);  // make a 16 byte ArrayBuffer
var bytes = new Uint8Array(b);  // make a Uint8Array view into ArrayBuffer
var longs = new Uint32Array(b);  // make a Uint32Array view into ArrayBuffer
var floats = new Float32Array(b);  // make a Float32Array view into ArrayBuffer

// print the contents of the views
console.log(bytes);
console.log(longs);
console.log(floats);

// change a byte using one of the views
bytes[1] = 255;

// print the contents again
console.log(bytes);
console.log(longs);
console.log(floats);

Copy and paste all that code into your JavaScript console. You should see something like

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 
[0, 0, 0, 0] 
[0, 0, 0, 0] 
[0, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 
[65280, 0, 0, 0] 
[9.147676375112406e-41, 0, 0, 0] 

Note: using multiple views of different types on the same array buffer is NOT cross platform compatible. In other words, you'll get different results on a big endian platform vs a little-endian platform. At the moment there are no popular big endian platforms with browsers that support TypedArrays so you can kind of ignore this issue though your page may break on some future platform. If you want to read/write data in a platform independent way you should use a DataView. Otherwise, the main point of using multiple views on the same buffer is to upload packed vertex data for example float positions packed with uint32 RGBA colors. In that case, it will work cross platform because you aren't reading/writing the same data with the views.

As pointed out, JS native arrays and TypedArrays are not related except in that you can use a JS native array to initialize a TypedArray

var jsArray = [1, 2, 3, 4];
var floats = new Float32Array(jsArray);  // this is a new array, not a view.

console.log(jsArray);
console.log(floats);

jsArray[1] = 567;  // effects only the JS array

console.log(jsArray);
console.log(floats);

floats[2] = 89;  // effects only the float array

console.log(jsArray);
console.log(floats);

Pasting into console I get

[1, 2, 3, 4] 
[1, 2, 3, 4] 
[1, 567, 3, 4] 
[1, 2, 3, 4] 
[1, 567, 3, 4] 
[1, 2, 89, 4] 

Note that you can get the underlying ArrayBuffer from any typed array.

var buffer = floats.buffer;

And create new views

var longs = new Uint8Array(buffer);
console.log(longs);

// prints [0, 0, 128, 63, 0, 0, 0, 64, 0, 0, 178, 66, 0, 0, 128, 64] 

You can also create views that cover a portion of the buffer.

var offset = 8; // Offset is in bytes
var length = 2; // Length is in units of type

// a buffer that looks at the last 2 floats
var f2 = new Float32Array(buffer, offset, length); 
console.log(f2);

// prints [89, 4]

As for textures and typed arrays here's a snippet using a Float32Array to update a Floating point texture.

main();
function main() {
  var canvas = document.getElementById("canvas");
  var gl = canvas.getContext("webgl");
  if (!gl) {
    alert("no WebGL");
    return;
  }
  var f = gl.getExtension("OES_texture_float");
  if (!f) {
    alert("no OES_texture_float");
    return;
  }

  var program = twgl.createProgramFromScripts(
    gl, ["2d-vertex-shader", "2d-fragment-shader"]);
  gl.useProgram(program);

  var positionLocation = gl.getAttribLocation(program, "a_position");
  var resolutionLocation = gl.getUniformLocation(program, "u_resolution");
  gl.uniform2f(resolutionLocation, canvas.width, canvas.height);

  var buffer = gl.createBuffer();
  gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
  gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
    -1, -1, 1, -1, -1, 2,
    -1,  1, 1, -1,  1, 1]), gl.STATIC_DRAW);
  gl.enableVertexAttribArray(positionLocation);
  gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);

  var tex = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, tex);
  var width = 64;
  var height = 64;
  var pixels = new Float32Array(width * height * 4);
  for (var y = 0; y < height; ++y) {
    for (var x = 0; x < width; ++x) {
      var offset = (y * width + x) * 4;
      pixels[offset + 0] = (x * 256 / width) * 1000;
      pixels[offset + 1] = (y * 256 / height) * 1000;
      pixels[offset + 2] = (x * y / (width * height)) * 1000;
      pixels[offset + 3] = 256000;
    }
  }
  gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.FLOAT,
                pixels);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
  gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

  function randInt(range) {
    return Math.floor(Math.random() * range);
  }

  function render() {
    // update a random pixel
    var x = randInt(width);
    var y = randInt(height);
    var offset = (y * width + x) * 4;
    pixels[offset + 0] = randInt(256000);
    pixels[offset + 1] = randInt(256000);
    pixels[offset + 2] = randInt(256000);

    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.FLOAT,
                  pixels);

    gl.drawArrays(gl.TRIANGLES, 0, 6);
    requestAnimationFrame(render);
  }
  render();
}

<script src="https://twgljs.org/dist/2.x/twgl.min.js"></script>
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
void main() {
    gl_Position = vec4(a_position, 0, 1);
}
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
uniform vec2 u_resolution;
uniform sampler2D u_tex;
void main() {
    vec2 texCoord = gl_FragCoord.xy / u_resolution;
    vec4 floatColor = texture2D(u_tex, texCoord);
    gl_FragColor = floatColor / 256000.0;  
}
</script>
<canvas id="canvas" width="400" height="300"></canvas>

All of that suggests the issue you are seeing is maybe related to something else? As for debugging, as pointed out above ArrayBuffers are very straight forward, there's no lazy buffering or anything. So if you want to see into an ArrayBuffer make a view for it so the debugger has some way to know what you want to be displayed.