Reading from multiple render targets in DirectX

Question

I have two render target views, my back buffer and a Texture2D that represents a specialized mask (I can't use the stencil buffer for what I need). My render method looks roughly like this:

// clear the render target and depth stencil views, set render targets, 
// update subresources, set vertex buffers, topology, input layout, etc.

// draw to the back buffer and the mask
m_d3dContext->PSSetShader(m_pixelShader1.Get(), nullptr, 0);
m_d3dContext->IASetIndexBuffer(m_indexBuffer1.Get(), DXGI_FORMAT_R16_UINT, 0); 
m_d3dContext->DrawIndexed(m_indexCount1, 0, 0);

// read from the mask and draw to the back buffer
m_d3dContext->PSSetShader(m_pixelShader2.Get(), nullptr, 0);
m_d3dContext->IASetIndexBuffer(m_indexBuffer2.Get(), DXGI_FORMAT_R16_UINT, 0);
m_d3dContext->DrawIndexed(m_indexCount2, 0, 0);

PixelShader1 looks like this (leaving out the input struct and some other parts):

struct PixelShaderInput
{
    float4 Color   : COLOR;
    float2 Texture : TEXCOORD;
};

struct PixelShaderOutput
{
    float4 Color : SV_TARGET0;
    float4 Mask  : SV_TARGET1;
};

PixelShaderOutput main(PixelShaderInput input)
{
    PixelShaderOutput output; 
    output.Color.rgba = input.Color;
    output.Mask.rgba = float4(1, 0, 0, 1);
    return output;
}

PixelShader2 looks like this:

struct PixelShaderInput
{
    float3 Color   : COLOR0;
    float2 Texture : TEXCOORD0;
    float4 Mask    : COLOR1;
};

struct PixelShaderOutput
{
    float4 Color : SV_TARGET0; 
};

PixelShaderOutput main(PixelShaderInput input)
{
    PixelShaderOutput output; 

    // do some work with input.Texture

    if (input.Mask.x == 0)
    {
        output.Color = float4(0, 0, 0, 1); }
    }
    else
    {
        output.Color = float4(1, 1, 1, 1); 
    }

    return output;
}

Using Visual Studio's graphics debugging tools, I can see that the mask texture is written correctly by PixelShader1. At the end of the frame, there is red geometry appearing in the right positions. Further the output to the back buffer (which looks different) is also as expected. So I strongly believe that PixelShader1 is correct.

However, I am getting unexpected results from PixelShader2. Stepping through it, the values for input.Mask.x are all over the place. One would expect them to be either 1 or 0, as that is all the other pixel shader sets them to be, but instead the values look an awful lot like the texture coordinates.

So, first of all, is what I'm trying to do even possible? If it is, can anybody spot my error in how I'm reading COLOR1? I have been stumped by this for hours, so any help would be greatly appreciated.

I'm working in DirectX 11, shader model 4.0 level 9_1, and vc110.

Answer 1

You have a few concepts wrong in how your PixelShader2 is written. PixelShader1 writes to two render textures the values of Color and Mask. In PixelShader 2 you will need to read these values from textures, and not as vertex input.

Texture2D<float4> MaskTexture;
SamplerState MaskSampler;

struct PixelShaderInput
{
    float3 Color   : COLOR0;
    float2 Texture : TEXCOORD0;
};

struct PixelShaderOutput
{
    float4 Color : SV_TARGET0; 
};

PixelShaderOutput main(PixelShaderInput input)
{
    PixelShaderOutput output; 

    // do some work with input.Texture

    float maskValue = MaskTexture.Sample(MaskSampler, Texture).x;

    if (maskValue == 0)
    {
        output.Color = float4(0, 0, 0, 1);
    }
    else
    {
        output.Color = float4(1, 1, 1, 1); 
    }

    return output;
}

So in order to do that you will need to pass in MaskTexture (this is your second render target output from PixelShader1) as a ShaderResource, via SetShaderResources(). You will also need to create a SamplerState and set that into the device also via SetSamplerStates().