fragment shader blending and gl_LastFragColor undeclared identifier on android 2.3, OPENGL ES 2.0

Question

Main Questions:

1) How can I get the fragment shader to not update the current pixels to a transparent pixel ONLY if destination color is not transparent?

2) Does gl_LastFragColor work for android 2.3 OpenGL ES 2.0?

Background:

I am trying to write text with OpenGL ES 2.0 using batch texturing. The challenge in the fragment shader is to only update the pixel colors for the text if the destination pixel color has alpha of 1, which I know it means that the pixel doesn't belong to the text.

The texture of the text atlas is stored in a bitmap: Bitmap.Config.ALPHA_8

After several hours I noticed three problems:

First problem: Somehow alpha of 1 is being used as opaque and 0 as transparent.

Second problem: Suppose I display "Hi". Although my draworder is supposed to draw "Hi" by texturing H, then i, it is doing it as: "i", then "H".

I know this because if I manually reverse the order of the vertices for "Hi" to start from "i", then I get my text. This is a problem because of problem 3.

Third problem: I believe this is the main problem. Say the letter H has texture box of exactly the width of 0.5, but I use texture of width 1, where the first 0.25 is white with alpha 1, the next 0.5 is white with alpha 0, last 0.25 is white with alpha 1. Same for "i".

When "i" is textured first, then "H" is textured second, the widths will intersect (on purpose, so "DD" and "dd" have different combined widths for example). But after "H" is rendered after "i", it is updating all the pixels across its width to be transparent, and most of the letter o disappears. This is correct behavior if "i" was rendered first then "H" and blending is disabled.

So I tried to use gl_LastFragColor to check the previous pixel before updating it but I got: Undeclared Identifier. So I am stuck with this fragment code:

Fragment code:

vec4 myTexture;
myTexture = texture2D(u_Texture, v_TexCoordinate);

if( myTexture[3] < 0.01){ // The texture is Bitmap.Config.ALPHA_8. If alpha is 0 . . .
    gl_FragColor = vec4(0.0,0.0,0.0,0.0);  // 0.0 is somehow transparent.
}
else{
    gl_FragColor = vec4(1.0,1.0,1.0,1.0); // display text in white. Somehow 1.0 is opaque.
    gl_FragColor = myTexture;

}

I am calling draw as:

    GLES20.glEnable (GLES20.GL_BLEND);
    GLES20.glBlendFunc (GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
    //      GLES20.glBlendFunc (GLES20.GL_ONE_MINUS_SRC_ALPHA, GLES20.GL_SRC_ALPHA); This makes alpha of 1 transparent!!


    GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, buffers[2]);
    GLES20.glDrawElements(GLES20.GL_TRIANGLES, drawOrderLength, GLES20.GL_UNSIGNED_SHORT, 0);

    GLES20.glDisable (GLES20.GL_BLEND);

P.s. If I switch alphas from 1 to 0, or equivalent trick, I get the same problem, with transparency inverted.

Answer 1

I think you can do what you're trying to achieve without using gl_LastFragColor (which, as Andon pointed out, is not part of standard OpenGL ES).

I would look into blend functions that use destination alpha. You may have to play with it to find out what exactly meets your requirements, but start with something like:

glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ONE);

This will leave the framebuffer unchanged for any pixels that already have an alpha value of 1.0 in the framebuffer, and replace the framebuffer pixel with your current fragment if the alpha value in the framebuffer is 0.0.

To use destination alpha, pick a framebuffer configuration that has alpha planes when you create your GLSurfaceView. Adding this before calling setRenderer will take care of this:

setEGLConfigChooser(8, 8, 8, 8, 16, 0);

Also make sure that you use 0.0 as the clear value for alpha in your glClearColor call.

To make sure that transparent parts of your texture are not draw, what you're doing by setting gl_FragColor to completely transparent should work with this. Another option is that you discard those fragments in the shader.

Answer 2

That is not going to work portably.

gl_LastFragColor is something that NV and Apple added through (pretty much identical) extensions.

Apple's extension is called GL_APPLE_shader_framebuffer_fetch, and you're not going to have said extension on Android. NV has the same extension, just replace APPLE with NV and make sure you have the necessary Tegra GPU.

Assuming you feel alright limiting yourself exclusively to Android devices that run NV Tegra GPUs, you may be able to get this functionality in your GLSL shader by placing the following line at the top of the shader:

#extension GL_NV_shader_framebuffer_fetch : require

If and when other vendors support this functionality, they will likely do so using GL_EXT_shader_framebuffer_fetch. The bottom line, however, is that this is an extension, and not widely available on anything other than iOS.

Answer 3

I figured it out. I saw here a good explanation using discard.

My final code is:

void main()
{
vec4 myTexture;
myTexture = texture2D(u_Texture, v_TexCoordinate);

if( myTexture[3] < 0.01){
    discard;
}
else{
    gl_FragColor = myTexture;

}

}

Discard will actually not write over the pixels.