How do I intersect two smooth shapes in OpenGL?

Question

I've successfully drawn two smooth shapes in OpenGL using a routine that generates a triangle strip whose outermost edge line has all its vertices at alpha 0. Now I want to intersect them, but I always seem to lose one shape's smooth edges. Here's the code I'm using:

// Draw: smooth black shape as normal

glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_TRUE);
// Draw: smooth black shape into alpha channel

glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glBlendFunc(GL_DST_ALPHA, GL_SRC_ALPHA);
// Draw: Yellow overlay shape with black shape's alpha

// Reset blending modes for rest of program

And here's the result (at bottom) — the yellow shape loses its smooth right-hand edge because the alpha in those pixels is now 1. How can I get a smooth intersected shape?

Answer 1

I struggled with this problem for a long time and tried every drawing order and combination of glColorMask and glBlendEquation. In the end I realised there's a very simple solution — premultiplied alpha.

In my 'smooth shape' drawing routine, rather just than drawing the outer smoothing edge as same colour, alpha=0, I allowed the outer smoothing colour to be specified. For the yellow shape I specified the black colour, and that gave me a smooth edge at the same time as the overall shape having a smooth edge — even though the yellow shape's right-hand edge isn't masked by the alpha channel.

Answer 2

You need something like this snippet:

glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glBlendEquation(GL_FUNC_ADD);
glEnable(GL_BLEND);

I think that a key element is glEnable(GL_BLEND).