WebGL Multitexturing with Video

Question

I'm trying to use two textures in a fragment shader and having trouble getting WebGL to send the textures properly. I can send one at a time, but when I try to do both at once, I just get black. I've seen some other multitexturing examples around, but all of them deal with loading an array of images.

I want to load a video into one texture, and a canvas into the 2nd texture. I'm fairly certain all my shaders are fine, since I'm just moving them over from a c++ opengl program I wrote. Also, I can display the video or canvas each separately by commenting one or the other out, but like I mentioned above, together they seem to trigger an error.

Here's the snippet of code where I'm creating and filling the textures.

var texture1 = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture1);

var texture2 = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture2);

gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MAG_FILTER, gl.NEAREST);

And then inside my draw loop:

gl.texImage2D(gl.TEXTURE_2D,0,gl.RGBA,gl.RGBA,gl.UNSIGNED_BYTE, video);
var tex1loc = gl.getUniformLocation(program,"u_image");
gl.uniform1i(tex1loc, 0);
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, texture1);

gl.texImage2D(gl.TEXTURE_2D,0,gl.RGBA,gl.RGBA,gl.UNSIGNED_BYTE, canvas);
var tex2loc = gl.getUniformLocation(program, "u_image2");
gl.uniform1i(tex2loc, 1);
gl.activeTexture(gl.TEXTURE1);
gl.bindTexture(gl.TEXTURE_2D, texture2);

I'm also getting a warning message saying:

GL_INVALID_ENUM : glActiveTexture: texture was GL_FALSE

GL_INVALID_ENUM : glActiveTexture: texture was GL_LINES

and also:

WebGL: drawArrays: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering or is not 'texture complete'. Or the texture is Float or Half Float type with linear filtering while OES_float_linear or OES_half_float_linear extension is not enabled.

The line triggering the warning is:

gl.drawArrays(gl.TRIANGLES, 0,6);

Thanks in advance for the help!

Answer 1

gl.activeTexture sets the texture unit all other texture commands effect. For each texture unit there are 2 bind points, TEXTURE_2D and TEXTURE_CUBE_MAP.

You can think of it like this

gl = {
  activeTextureUnit: 0,
  textureUnits: [
     { TEXTURE_2D: null: TEXTURE_CUBE_MAP: null, },
     { TEXTURE_2D: null: TEXTURE_CUBE_MAP: null, },
     { TEXTURE_2D: null: TEXTURE_CUBE_MAP: null, },
     ...
  ],
};

gl.activeTexture just does this

gl.activeTexture = function(unit) {
 gl.activeTextureUnit = unit - gl.TEXTURE0;
};

gl.bindTexture does this

gl.bindTexture = function(bindPoint, texture) {
  gl.textureUnits[gl.activeTextureUnit][bindPoint] = texture;
};

gl.texImage2D and gl.texParamteri look up which texture to work with like this

gl.texImage2D = function(bindPoint, .....) {
  var texture = gl.textureUnits[gl.activeTextureUnit][bindPoint];
  // now do something with texture

so, knowing all that, the first issue with the code is it creates 2 textures, texture1 and texture2 but then it only sets the texture parameters for texture2 because the second call to gl.bindTexture sets the texture that is bound to the current activeTexture on its TEXTURE_2D bind point and then gl.texParameteri only operates on that texture.

It needs to set the parameters for texture1 before it binds texture2.

Similarly the draw code need to call gl.activeTexture and/or gl.bindTexture before it calls gl.texImage2D otherwise it is operating on whatever texture is bound to bind point TEXTURE_2D on whatever is the currently active texture unit.

As for your warnings related to gl.activeTexture are you sure you posted all the code related to textures? If you posted all of it those warnings don't make a lot of sense since your only calls to gl.activeTexture look valid.

Otherwise the other warning about the texture on unit 0 not being renderable is because you didn't set the parameters for texture1 as explained above.

So, to be clear, your code should look like this

var texture1 = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture1);

// you need to set parameters for texture1
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MAG_FILTER, gl.NEAREST);

var texture2 = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture2);

gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D,gl.TEXTURE_MAG_FILTER, gl.NEAREST);

// these should be looked up at init time
var tex1loc = gl.getUniformLocation(program,"u_image");
var tex2loc = gl.getUniformLocation(program, "u_image2");

draw loop:

// call active texture first
gl.activeTexture(gl.TEXTURE0);
// then bind a texture. This now binds texture1 to unit 0
gl.bindTexture(gl.TEXTURE_2D, texture1);
gl.texImage2D(gl.TEXTURE_2D,0,gl.RGBA,gl.RGBA,gl.UNSIGNED_BYTE, video);
gl.uniform1i(tex1loc, 0);

// call active texture first
gl.activeTexture(gl.TEXTURE1);
// then bind a texture. This now binds texture2 to unit 1
gl.bindTexture(gl.TEXTURE_2D, texture2);
gl.texImage2D(gl.TEXTURE_2D,0,gl.RGBA,gl.RGBA,gl.UNSIGNED_BYTE, canvas);
gl.uniform1i(tex2loc, 1);