Database for Importing NUnit results?

Question

I have a large set of NUnit tests; I need to import the results from a given run into a database, then characterize the set of results and present them to the users (email for test failures, web presentation for examining results). I need to be tracking multiple runs over time, as well (for reporting failure rates over time, etc.).

The XML will be the XML generated by nunit-console. I would like to import the XML with a minimum of fuss into some database that can then be used to persist and present results. We will have a number of custom categories that we will need to be able to sort across, as well.

Does anyone know of a database schema that can handle importing this type of data that can be customized to our individual needs? This type of problem seems like it should be common, and so a common solution should exist for it, but I can't seem to find one. If anyone has implemented such a solution before, advice would be appreciated as well.

Answer 1

It sounds to me like you're actually after a build server such as CruiseControl.NET or TeamCity.

Get the build server to run the tests, and it does the job of telling people what failed, and why.

I recommend TeamCity as it's several orders of magnitude easier to set up.

Answer 2

I am here looking to solve the same issue. We are currently leaning toward writing an XSLT to transform the XML results into insert statements. Then run the resulting file of insert statements through a command line SQL interpreter. Ideally, I would rather have an NUnit add-in/extention that handles all this for me. Unfortunately I have not been able to find one.

Answer 3

To build off IainMH's answer you may want to take a look at using Trac with BITTEN, it is an open source build system, that can run n-unit tests and report the results. I currently use it for that exact functionality.

Answer 4

When using MS SQL, you can import all XMLs to a common column of [xml] datatype. Upon this, xpaths, searching and transformations can be performed.

Answer 5

Another Alternative to CruiseControl or TeamCity is Atlassians Bamboo if you're strapped for cash. I'm a huge fan of their software for it's ease of use and they have a deal on where you can get bamboo for 10 bucks.

Answer 6

We've hoped to avoid this, but we've generated a database schema from the NUnit result XML schema; it's a bit deficient, however, because NUnit does some (inaccurate and strange) processing to determine some of the critical statistics ("ignored" vs. "not run", for example).

We're still hoping to find a schema / process that is NOT a complete CIT build system which can allow us to customize a database for importing the results, but currently we're using a hand-rolled database which we'll need to do a lot of customizing on to get the desired reporting.

Answer 7

Why do you need to have the results in a database? Who is going to use them? The number of failures cannot be large. If it is (repeatedly) your development process is wrong. Fix the process. Eliminate waste (one of the lean principles), don't collect it.

Take smaller steps (shorter iterations, continuous build), eliminate dependencies.

This is is not commonly done, because projects that have these kind of problems don't deliver but get cancelled (eventually).

[edit] Michael, tracking nunit failures over a longer time provides zero value. You need a short feedback loop. Fix the problems now. If you wait till you have accumulated a lot of problems, you are going to be overwhelmed by the noise.

Good problem tracking is done at the right (highest possible abstraction) level. Definitely not unit test.