Creating and blending a dynamic texture in OpenGL

Question

I need to render a sphere to a texture (done using a Framebuffer Object (FBO)), and then alpha blend that texture with the back buffer. So far I'm not doing any processing with the texture except clearing it at the beginning of every frame.

I should say that my scene consists of nothing but a planet in empty space, the sphere should appear next to or around the planet (kind of like a moon for now). When I render the sphere directly to the back buffer, it displays correctly; but when I do the intermediary step of rendering it to a texture and then blending that texture with the back buffer, the sphere only shows up when it is in front of the planet, the part that isn't in front is just "cut off":

I render the sphere using glutSolidSphere to a RGBA8 fullscreen texture that's bound to an FBO, making sure that every sphere pixel receives an alpha value of 1.0. I then pass the texture to a fragment shader program, and use this code to render a fullscreen quad - with the texture mapped onto it - to the backbuffer while alpha blending:

glBindFramebuffer(GL_FRAMEBUFFER, 0);
glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
glEnable(GL_BLEND);

glDisable(GL_DEPTH_TEST);
glMatrixMode(GL_MODELVIEW);
glPushMatrix();
glLoadIdentity();
glMatrixMode(GL_PROJECTION);
glPushMatrix();
glLoadIdentity();
glBegin(GL_QUADS);
glTexCoord2i(0, 1);
glVertex3i(-1,  1, -1);   // TOP LEFT
glTexCoord2i(0, 0);
glVertex3i(-1, -1, -1);   // BOTTOM LEFT
glTexCoord2i(1, 0);   
glVertex3i( 1, -1, -1);   // BOTTOM RIGHT
glTexCoord2i(1, 1);   
glVertex3i( 1,  1, -1);   // TOP RIGHT
glEnd();
glPopMatrix();
glMatrixMode(GL_MODELVIEW);
glPopMatrix();
glEnable(GL_DEPTH_TEST);

glDisable(GL_BLEND);

This is the shader code (taken from an FX file written in Cg):

sampler2D BlitSamp = sampler_state
{
    MinFilter = LINEAR;
    MagFilter = LINEAR;
    MipFilter = LINEAR;
    AddressU = Clamp;
    AddressV = Clamp;
};

float4 blendPS(float2 texcoords : TEXCOORD0) : COLOR
{
    float4 outColor = tex2D(BlitSamp, texcoords);
    return outColor;
}

I don't even know whether this is a problem with the depth buffer or with alpha blending, I've tried a lot of combinations of enabling and disabling depth testing (with a depth buffer attached to the FBO) and alpha blending.

EDIT: I tried just rendering a blank fullscreen quad straight to the back buffer and even that was cropped around the planet's edges. For some reason, enabling depth testing for rendering the quad (that is, removing the lines glDisable(GL_DEPTH_TEST) and glEnable(GL_DEPTH_TEST) in the code above) got rid of the problem, but now everything but the planet and the sphere appears white:

I made sure (and could confirm) that the alpha channel of the texture is 0 at every pixel but the sphere's, so I don't understand where the whiteness could be introduced. (Would also still be interested in an explanation why enabling depth testing has this effect.)

Answer 1

I solved it or at least came up with a work around.

First off, the whiteness stems from the fact that glClearColor had been set to glClearColor(1.0f, 1.0f, 1.0f, 1000.0f), so everything but the planet wasn't even written to in the end. I now copy the contents of the back buffer (which is the planet, the atmosphere, and the space around it) to the texture before rendering the sphere, and I render the atmosphere and space before that copy/blit operation, so they are included in it. Previously, everything but the planet itself was rendered after my quad, which - when using depth testing - apparently placed everything behind the quad, making it invisible.

The reference implementation of the effect I'm trying to achieve has always used this kind of blit operation in its code but I didn't think it was necessary for the effect. Now I feel like there might be no other way...

Answer 2

I see two possible sources of error here:

1. Rendering to the FBO

If the missing pixels are not even present in the FBO after rendering, there must be some mechanism which discarded the corresponding fragments. The OpenGL pipeline includes four different types of fragment tests which can lead to fragments being discarded:

1. Scissor Test: Unlikely to be the cause, as the scissor test only affects a rectangular portion of the screen.
2. Alpha Test: Equally unlikely, as your fragments should all have the same alpha value.
3. Stencil Test: Also unlikely, unless you use stencil operations when drawing the background planet and copy over the stencil buffer from the back buffer to the FBO.
4. Depth Test: Same as for stencil test.

So there's a good chance that rendering into FBO is not the issue here. But just to be absolutely sure, you should read back your color attachment texture and dump it into a file for inspection. You can use the following function for that:

void TextureToFile(GLuint texture, const char* filename) {
  glBindTexture(GL_TEXTURE_2D, texture);
  GLint width, height;
  glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_WIDTH, &width);
  glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_HEIGHT, &height);

  std::vector<GLubyte> pixels(3 * width * height);
  glGetTexImage(GL_TEXTURE_2D, 0, GL_RGB, GL_UNSIGNED_BYTE, &pixels[0]);

  std::ofstream out(filename, std::ios::out | std::ios::binary);
  out << "P6\n"
      << width << '\n'
      << height << '\n'
      << 255 << '\n';
  out.write(reinterpret_cast<const char*>(&pixels[0]), pixels.size());  
}

The resulting file is a portable pixmap (.ppm). Be sure to unbind the FBO before reading back the texture.

2. Texture mapping

Assuming rendering into the FBO works as expected, the only other source of error is blending the texture over the previously rendered scene. There are two scenarios:

a) Fragments get discarded

The possible reasons for fragments to get discarded are the same as in 1.:

1. Scissor Test: Nope, affects rectangular areas only.
2. Alpha Test: Probably not, the texels covered sphere should all have the same alpha value.
3. Stencil Test: Might be the cause if you use stencil operations/stencil testing when drawing the background planet and the old stencil state is still active.
4. Depth Test: Might be the cause, but as you already disable it, it really shouldn't have any effect.

So you should make sure that all of these tests are disabled, especially the stencil test.

b) Wrong results from blending

Assuming all fragments reach the back buffer, blending is the only thing which could still cause the wrong result. With your blending function (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) the values in the back buffer are irrelevant for blending, and we assume that the alpha values in the texture are correct. So I see no reason for why blending should be the root cause here.

Conclusion

In conclusion, the only sensible cause for the observed result seems to be stencil testing. If it's not, I'm out of options :)