Running all NUnit tests in all assemblys from Cygwin command line

Question

I'm trying to create a short bash script someone could run in cygwin to execute all nunit tests in a .NET project using nunit-console. Currently i have the system version of nunit aliased to "nunit" so if i run "nunit" it will execute nunit-console.

My first step was to try and find all the test assemblies recursively. This MOSTLY works:

find . -name *.Test*.dll

but it returns both /obj and /bin versions of a dll.

Second i need to figure out a way to pass all of the results from that find to nunit, preferably in one execution which so far I have no clue how to do.

Any cygwin / bash gurus out there that can help?

Answer 1

Does the list of assemblies change dynamically? If it doesn't change often then you probably don't need to determine what assemblies to run on at runtime. Rather, your problem is better solved through the use of an NUnit project. For example, create the file tests.nunit, with contents:

<NUnitProject>
  <Settings activeconfig="Debug"/>
  <Config name="Debug">
    <assembly path="ProjectA\bin\Debug\App.dll"/>
    <assembly path="ProjectB\bin\Debug\Widget.dll"/>
  </Config>
  <Config name="Release">
   <assembly path="ProjectA\bin\Release\App.dll"/>
    <assembly path="ProjectB\bin\Release\Widget.dll"/>
  </Config>
</NUnitProject>

Running nunit-console tests.nunit would run all the tests in the App and Widget assemblies, in the Debug folder. You can also omit the Config and activeconfig stuff if it's not relevant and just list assemblies that will always be tested regardless of active configuration.

Check out the NUnit documentation on running on multiple assemblies.

Answer 2

Here is my working script for this

check_err()
{
    # parameter 1 is last exit code
    # parameter 2 is error message that should be shown if error code is not 0
    if [ "${1}" -ne "0" ]; then
        cat '~temp.log'
        echo ${2}
        rm -f '~temp.log' > /dev/null
        exit ${1}
    fi;
    rm -f '~temp.log' > /dev/null
}

for i in `find . -print | grep 'bin/Debug/[^/]*\.Tests\.dll$'`; do
    echo "Running tests from ${i} with NUnit...";
    cmd.exe /c "Lib\NUnit\nunit-console.exe ${i} /framework:net-4.0" > '~temp.log'
    check_err $? "Some tests failed."
done

But there's a subtle problem. NUnit sometimes returns negative error codes which is OK for windows, but illegal for *nix. I don't know how Cygwin works with negative exit codes but MinGW32 just treats them as 0, i.e. "$?" == "0" as a result.

Good news, negative error codes are somewhat rare for NUnit and are indicating problems with running NUnit itself (see this question for details). I didn't find a workaround for this problem, so I have negative error code check only on build server. Locally from bash console I'm handing only positive ones.

Answer 3

Something like this:

find . -name "*.Test*.dll" -path /path/to/skip -prune -o -exec nunit {} \;

Untested - try it like this first to see if it gives you the correct files:

find . -name "*.Test*.dll" -path /path/to/skip -prune -o -print