Trapping Error Status in MSBuild

Question

As part of some build automation of running xUnit.net tests with MSBuild, I'm running into a case where I need to loop over a batch of items.

Inside the loop, I need to detect whether an iteration failed, but I want to continue executing regardless. Then after the batched bit, I need to know whether one or more errors have occurred in order to report the outcome to TeamBuild.

IOW, in pseudocode:

Task Name=RunTests
  CreateItems
  ForEach item CallTarget Target=RunTest ContinueOnError=true
  CombineNUnitResults
  Report success/failure

Task Name=RunTest
   XUnit item

I'm hoping this can be achieved without a custom task (or hacking the xunit.net MSBuild task as Jonne did). (But willing to use MSBuild Community or Sdc tasks)

And @BradWilson: I this is isnt possible to do cleanly, I'll be looking for Jonne's change a la the NUnit task to also make it into the xunit task

See also: How do I get Team Build to show test results and coverage for xUnit.net test suite?

Answer 1

Go grab 1.5 Beta. We fixed this by introducing the ExitCode output parameter to our xunit MSBuild task!

http://xunit.codeplex.com/Release/ProjectReleases.aspx

Answer 2

This is what we do:

<NUnit Assemblies="@(TestAssemblies)"
    ToolPath="$(NUnitPath)"
    WorkingDirectory="%(TestAssemblies.RootDir)%(TestAssemblies.Directory)"
    OutputXmlFile="@(TestAssemblies->'%(FullPath).$(NUnitFile)')"
    Condition="'@(TestAssemblies)' != ''"
    ExcludeCategory="$(ExcludeNUnitCategories)"
    ContinueOnError="true">
  <Output TaskParameter="ExitCode" ItemName="NUnitExitCodes"/>
</NUnit>

<Error Text="Test error(s) occured" Code="%(NUnitExitCodes.Identity)" Condition=" '%(NUnitExitCodes.Identity)' != '0' And '@(TestAssemblies)' != ''"/>

This will run all the unit tests regardless of failure but will fail after all have been run if there were any failures. Note ContinueOnError="true" ensures that they are all run and the Error at the end checks to see if any of them failed (in nunit 0 indicates success, anything else is a failure).

Note: this is using the MSBuildCommunityTasks NUnit task but if you're just using exec with the nunit exe, you can get the same effect. The output "ExitCode" is common to any Task that inherits from ToolTask.