Build are not tagged as unstable when unit test fails

Question

I have added a new job in my hudson server which builds the project with a makefile.

Execute shell command:

#!/bin/bash
cd $JOB_NAME
make

My makefile looks like this

SDK_31 = iphonesimulator3.1
TARGET_DEV = myProject
TARGET_TEST = unitTest

all: debug
debug:
        xcodebuild -sdk ${SDK_31} -target "${TARGET_DEV}" -configuration Debug
        xcodebuild -sdk ${SDK_31} -target "${TARGET_TEST}" -configuration Debug

clean:
        xcodebuild -alltargets clean
        rm -rf build

But when hudson build the projects, some unit tests fail but the build is tagged as successful.

What should I have to do to have an "unstable project" ?

Best regards,

Answer 1

You should configure Hudson to record unit test results, by enabling the 'Publish Junit test result report' post-build action.

post-build actions http://img141.imageshack.us/img141/5136/hudsonjunit.png

---

Update: If you can't get JUnit XML output, you should be able to use the Text-finder plugin to change the build status:

This plugin lets you search keywords in the files you specified and use that to mark the build as success or a failure.

Answer 2

I found this handy ruby script by Christian Hedin that converts the output of OCUnit tests (the format used by Xcode) into JUnit xml files (the format used by Hudson).

You can grab the script on github: http://github.com/ciryon/OCUnit2JUnit

and for an explanation of how to use it, here's his blog post about it: http://blog.jayway.com/2010/01/31/continuos-integration-for-xcode-projects/

Basically, you pipe xcodebuild into ocunit2junit.rb with a command like this:

/usr/bin/xcodebuild -target UnitTests | /usr/local/bin/ocunit2junit.rb

and it places the xml files into a test-reports folder that it creates at the root of your project folder. Then tell Hudson to copy the test-reports/*.xml artifacts as the JUnit results and you're set.

This setup will allow Hudson to correctly identify if a unit test has passed or failed and correctly mark the stability of the build.

I've been using it for a month now and it works great. The setup was very simple.