MonoGame: stencil buffer not working

Question

I'm trying to add shadows to my MonoGame-based 2D game. At first I just rendered semitransparent black textures to the frame, but they sometimes overlap and it looks nasty:

I tried to render all shadows into the stencil buffer first, and then use a single semitransparent texture to draw all shadows at once using the stencil buffer. However, it doesn't work as expected:

The two problems are:

• The shadows are rendered into the scene
• The stencil buffer is seemingly unaffected: the semitransparent black texture covers the entire screen

Here's the initialization code:

StencilCreator = new DepthStencilState
{
    StencilEnable = true,
    StencilFunction = CompareFunction.Always,
    StencilPass = StencilOperation.Replace,
    ReferenceStencil = 1
};

StencilRenderer = new DepthStencilState
{
    StencilEnable = true,
    StencilFunction = CompareFunction.Equal,
    ReferenceStencil = 1,
    StencilPass = StencilOperation.Keep
};

var projection = Matrix.CreateOrthographicOffCenter(0, GraphicsDevice.Viewport.Width, GraphicsDevice.Viewport.Height, 0, 0, 1); 
var halfPixel = Matrix.CreateTranslation(-0.5f, -0.5f, 0);

AlphaEffect = new AlphaTestEffect(GraphicsDevice)
{
    DiffuseColor = Color.White.ToVector3(),
    AlphaFunction = CompareFunction.Greater,
    ReferenceAlpha = 0,
    World = Matrix.Identity,
    View = Matrix.Identity,
    Projection = halfPixel * projection
};

MaskingTexture = new Texture2D(GameEngine.GraphicsDevice, 1, 1);
MaskingTexture.SetData(new[] { new Color(0f, 0f, 0f, 0.3f) });

ShadowTexture = ResourceCache.Get<Texture2D>("Skins/Common/wall-shadow");

And the following code is the body of my Draw method:

// create stencil
GraphicsDevice.Clear(ClearOptions.Stencil, Color.Black, 0f, 0);
var batch = new SpriteBatch(GraphicsDevice);
batch.Begin(SpriteSortMode.Immediate, null, null, StencilCreator, null, AlphaEffect);
foreach (Vector2 loc in ShadowLocations)
{
    newBatch.Draw(
        ShadowTexture,
        loc,
        null,
        Color.White,
        0f,
        Vector2.Zero,
        2f,
        SpriteEffects.None,
        0f
    );
}
batch.End();

// render shadow texture through stencil
batch.Begin(SpriteSortMode.Immediate, null, null, StencilRenderer, null);
batch.Draw(MaskingTexture, GraphicsDevice.Viewport.Bounds, Color.White);
batch.End();

What could possibly be the problem? The same code worked fine in my XNA project.

Answer 1

I worked around the issue by using a RenderTarget2D instead of the stencil buffer. Drawing solid black shadows to a RT2D and then drawing the RT2D itself into the scene with a semitransparent color does the trick and is much simplier to implement.