Hudson build successful with unit test failures

Question

This is related to another question I asked about trying to fail Hudson on a single unit test failure.

When I run Maven on my build standalone, it fails due to unit test errors (running mvn clean install) but when I run it in Hudson, it says there was an error due to test failure, yet the build still is successful (and marked as unstable). Any thoughts?

Answer 1

What type of Hudson job did you create? I suspect it's the m2 project rather than the freestyle project. In the m2 project, Hudson will actually tweak the way your pom is configured so as to prevent failures and instead mark it as unstable. This was intentional.

Try setting maven.test.failure.ignore=false under the Advanced setting as I believe Hudson automatically sets that to true.

Answer 2

Hudson's convention is that it will only look for tests if the build returned success. If the build failed hudson interprets that to mean there was some unrecoverable error that caused the build to not work, and not that there were some tests that failed.

It's quite useful differential as thus unstable (yellow) means the build completed and the tests were all able to run, but some failed.

And error means that the build failed in that it couldn't compile or that there was some other unrecoverable error that caused the tests not to run, in this case hudson won't report on the unit test state.

Answer 3

there are also hudson options in each job's config to determine how to act if a build becomes unstable, such as whether or not to send emails, etc.

Answer 4

just saw this defect is reported on the Hudson Bugtracker