MSBuild ItemGroup Include/Exclude pattern issue

Question

Problem: an ItemGroups array isn't correctly build based on the value passed in the exclude attribute.

If you run this scrip it creates some sample file then tries to create an array called TheFiles based on the Include/Exclude attributes, problem is when the Exclude is anything other than hardcoded or a very simple property it gets it wrong.

The target DynamicExcludeList's incorrectly selects these files:
.\AFolder\test.cs;.\AFolder\test.txt

The target HardcodedExcludeList's correctly selects these files:
.\AFolder\test.txt

Any help much appreciated, this is driving me nuts.

(note its msbuild v4)

    <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Run">

      <Target Name="Run" >
        <CallTarget Targets="CreateSampleFiles" />
        <CallTarget Targets="DynamicExcludeList" />
        <CallTarget Targets="HardcodedExcludeList" />
      </Target>

      <Target Name="CreateSampleFiles" >
        <MakeDir Directories="AFolder" />
        <WriteLinesToFile Lines="Test" File="AFolder\test.cs" Overwrite="true" />
        <WriteLinesToFile Lines="Test" File="AFolder\test.txt" Overwrite="true" />
      </Target>

      <Target Name="DynamicExcludeList" >

        <PropertyGroup>
          <CommonFileExclusion>.\DIRECTORY_NAME_TOKEN\**\*.cs</CommonFileExclusion>
          <FinalExcludes>$(CommonFileExclusion.Replace('DIRECTORY_NAME_TOKEN', 'AFolder'))</FinalExcludes>
        </PropertyGroup>

        <Message Text="FinalExcludes: $(FinalExcludes)" />
        <ItemGroup>
          <TheFiles 
            Include=".\AFolder\**\*;" 
            Exclude="$(FinalExcludes)"
          />
        </ItemGroup>
        <Message Text="TheFiles: @(TheFiles)" />

      </Target>

      <Target Name="HardcodedExcludeList" >

        <PropertyGroup>
          <FinalExcludes>.\AFolder\**\*.cs</FinalExcludes>
        </PropertyGroup>

        <Message Text="FinalExcludes: $(FinalExcludes)" />
        <ItemGroup>
          <TheFilesWithHardcodedExcludes
            Include=".\AFolder\**\*;"
            Exclude="$(FinalExcludes)"
          />
        </ItemGroup>
        <Message Text="TheFilesWithHardcodedExcludes: @(TheFilesWithHardcodedExcludes)" />

      </Target>  
    </Project>

This is the output, note the differences between 'TheFiles' and 'TheFilesWithHardcodedExcludes'

PS C:\SVN\TrunkDeployment\TestMsBuild> msbuild .\Test.build.xml
Microsoft (R) Build Engine Version 4.0.30319.1
[Microsoft .NET Framework, Version 4.0.30319.1]
Copyright (C) Microsoft Corporation 2007. All rights reserved.

Build started 8/10/2010 2:30:42 PM.
Project "C:\SVN\TrunkDeployment\TestMsBuild\Test.build.xml" on node 1 (default targets).
DynamicExcludeList:
  FinalExcludes: .\AFolder\**\*.cs
  TheFiles: .\AFolder\test.cs;.\AFolder\test.txt
HardcodedExcludeList:
  FinalExcludes: .\AFolder\**\*.cs
  TheFilesWithHardcodedExcludes: .\AFolder\test.txt
Done Building Project "C:\SVN\TrunkDeployment\TestMsBuild\Test.build.xml" (default targets).


Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.06

EDITS

I've updated the above script to use the CreateItem, however there is still an issue when the list of items to exclude contains more than 1 path (i.e. the value of CommonFileExclusion has changed):

    <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Run">

      <Target Name="Run" >
        <CallTarget Targets="CreateSampleFiles" />
        <CallTarget Targets="DynamicExcludeList" />
        <CallTarget Targets="HardcodedExcludeList" />
      </Target>

      <Target Name="CreateSampleFiles" >
        <MakeDir Directories="AFolder" />
        <WriteLinesToFile Lines="Test" File="AFolder\test.cs" Overwrite="true" />
        <WriteLinesToFile Lines="Test" File="AFolder\test.txt" Overwrite="true" />
        <WriteLinesToFile Lines="Test" File="AFolder\test.vb" Overwrite="true" />
      </Target>

      <Target Name="DynamicExcludeList" >

        <PropertyGroup>
          <CommonFileExclusion>.\DIRECTORY_NAME_TOKEN\**\*.cs;.\DIRECTORY_NAME_TOKEN\**\*.vb;</CommonFileExclusion>
          <FinalExcludes>$(CommonFileExclusion.Replace('DIRECTORY_NAME_TOKEN', 'AFolder'))</FinalExcludes>
        </PropertyGroup>

        <Message Text="FinalExcludes: $(FinalExcludes)" />
        <CreateItem Include=".\AFolder\**\*;"
                     Exclude="$(FinalExcludes)">
          <Output TaskParameter="Include" ItemName="TheFiles"/>
        </CreateItem>
        <Message Text="TheFiles: @(TheFiles)" />

      </Target>

      <Target Name="HardcodedExcludeList" >

        <PropertyGroup>
          <FinalExcludes>.\AFolder\**\*.cs;.\AFolder\**\*.vb</FinalExcludes>
        </PropertyGroup>

        <Message Text="FinalExcludes: $(FinalExcludes)" />
        <CreateItem Include=".\AFolder\**\*;"
                     Exclude="$(FinalExcludes)">
          <Output TaskParameter="Include" ItemName="TheFilesWithHardcodedExcludes"/>
        </CreateItem>
        <Message Text="TheFilesWithHardcodedExcludes: @(TheFilesWithHardcodedExcludes)" />

      </Target>
    </Project>

Answer 1

Ok, I tried a little and I think the problem comes from the fact that you use a property which represent SCALAR value for multiple values. I would recommend batching and transforming (see http://scottlaw.knot.org/blog/?p=402 and http://msdn.microsoft.com/en-us/library/ms171476.aspx). For example the following code is working :

<Target Name="DynamicExcludeList" >
  <ItemGroup>
    <ExtensionsExcluded Include="cs;vb" />
  </ItemGroup>

  <CreateItem Include=".\AFolder\**\*"
          Exclude="@(ExtensionsExcluded->'.\AFolder\**\*.%(identity)')">
    <Output TaskParameter="Include" ItemName="TheFiles"/>
  </CreateItem>
  <Message Text="TheFiles: @(TheFiles)" />
</Target>

Answer 2

In target DynamicExcludeList use CreateItem task instead of ItemGroup to dynamically generate your item :

<Target Name="DynamicExcludeList" >

  <PropertyGroup>
    <CommonFileExclusion>.\DIRECTORY_NAME_TOKEN\**\*.cs</CommonFileExclusion>
    <FinalExcludes>$(CommonFileExclusion.Replace('DIRECTORY_NAME_TOKEN', 'AFolder'))</FinalExcludes>
  </PropertyGroup>

  <Message Text="FinalExcludes: $(FinalExcludes)" />

  <CreateItem Include=".\AFolder\**\*;"
              Exclude="$(FinalExcludes)">
    <Output TaskParameter="Include" ItemName="TheFiles"/>
  </CreateItem>

  <Message Text="TheFiles: @(TheFiles)" />
</Target>

Theoretically ItemGroup and CreateItem are equivalent but I've seen case (dynamic situation) like this one when CreateItem must be use.