Where do Xcode Bots put their results, so I can parse them?

Question 1

I'd like to share the team's solution. We found the place where the Bot Results are stored, we parse it using bash, and send messages to the HUE light via curl system calls. We call the script in a scheme pre and post Build scripts.

We parse the bot's result plist at:

/Library/Server/Xcode/Data/BotRuns/Latest/output/xcodebuild_result.bundle/Info.plist

There you can find all sorts of cool data to use!:

<dict>
    <key>AnalyzerWarningCount</key>
    <integer>0</integer>
    <key>AnalyzerWarningSummaries</key>
    <array/>
    <key>ErrorCount</key>
    <integer>0</integer>
    <key>ErrorSummaries</key>
    <array/>
    <key>LogIdentifier</key>
    <string>705bffcb-7453-49ba-882f-80e1218b59cf</string>
    <key>LogPath</key>
    <string>1_Test/action.xcactivitylog</string>
    <key>Status</key>
    <string>IDEActionResultStatus_Succeeded</string>
    <key>TestFailureSummaries</key>
    <array/>
    <key>TestSummaryIdentifier</key>
    <string>a1554874-4d40-4e94-ae89-a73184ec97a9</string>
    <key>TestSummaryPath</key>
    <string>1_Test/action_TestSummaries.plist</string>
    <key>TestsCount</key>
    <integer>185</integer>
    <key>TestsFailedCount</key>
    <integer>0</integer>
    <key>WarningCount</key>
    <integer>0</integer>
    <key>WarningSummaries</key>
    <array/>
<dict>

AnalyzerWarningCount
ErrorCount
WarningCount
TestsFailedCount

Oh bash, my sometimes lover, come save the day again.

Also notice the use of the Plist Buddy for parsing Xcode's XML property list files. The primo choice for getting information in and out of plist files.

    #!/bin/bash
    #
    #   By Phil
    #
    exec > /tmp/my_log_file.txt 2>&1
    TEST_RESULT_PLIST="/Library/Server/Xcode/Data/BotRuns/Latest/output/xcodebuild_result.bundle/Info.plist"

    hue_light_green=false

    echo "testResultParse_OwlHue"

    #If not bot, return
    if [ "$(whoami)" != "_teamsserver" ]; then
        echo "$(whoami) - Not a bot!";
        exit 1
    fi

    #1 If file not found ERROR
    if [ ! -f $TEST_RESULT_PLIST ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":150,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "Test Result Plist not Found";
        exit 1
    fi

    #2 AnalyzerWarningCount BLUE
    AnalyzerWarningCount=$(/usr/libexec/PlistBuddy -c "Print :AnalyzerWarningCount" "${TEST_RESULT_PLIST}")
    if [ $AnalyzerWarningCount != 0 ]; then
        echo "AnalyzerWarningCount";
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"xy\":[0.16, 0.1],\"hue\":15815,\"sat\":255,\"effect\":\"none\",\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
    fi

    #3 WarningCount
    WarningCount=$(/usr/libexec/PlistBuddy -c "Print :WarningCount" "${TEST_RESULT_PLIST}")
    if [ $WarningCount != 0 ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"xy\":[0.58, 0.41],\"hue\":15815,\"sat\":255,\"effect\":\"none\",\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "WarningCount";
    fi

    #4 ErrorCount || TestsFailedCount ERROR
    ErrorCount=$(/usr/libexec/PlistBuddy -c "Print :ErrorCount" "${TEST_RESULT_PLIST}")
    if [ $ErrorCount != 0 ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":150,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "ErrorCount";
        exit 1
    fi

    #5 TestsFailedCount ERROR
    ErrorCount=$(/usr/libexec/PlistBuddy -c "Print :ErrorCount" "${TEST_RESULT_PLIST}")
    if [ $TestsFailedCount != 0 ]; then
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":150,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
        echo "TestsFailedCount";
        exit 1
    fi

    #6 None of the above.  SUCCESS
    if [ "$hue_light_green" = true ] ; then
        echo "SUCCESS";
        curl -X PUT -d "{\"on\":true,\"bri\":32,\"effect\":\"none\",\"hue\":25500,\"sat\":255,\"alert\":\"lselect\"}" ipaddress/api/testestest/lights/3/state
    fi

AnalyzerWarningCount Blue
ErrorCount Red
WarningCount Orange
TestsFailedCount Red

Now when we get a count for any of the above, we get a blinking color change. For example, the following produces a bright blue from our hue: enter image description here

Question 2

The result path for OS X Server 4.0 seems to be:

/Library/Developer/XcodeServer/IntegrationAssets/Your_Bot/

Archive.xcarchive.zip
build.log
buildService.log
Your_Bot.ipa
sourceControl.log
xcodebuild_result.bundle.zip

The xcodebuild_result.bundle is a zip file now, I parse the build result from buildService.log instead:

Build results summary: {
analyzerWarningChange = 14;
analyzerWarningCount = 14;
errorChange = 0;
errorCount = 0;
improvedPerfTestCount = 0;
regressedPerfTestCount = 0;
testFailureChange = 0;
testFailureCount = 0;
testsChange = 0;
testsCount = 0;
warningChange = 20;
warningCount = 20;
}

Question 3

To anyone looking here after these answers, the scraping of the buildService.log files is not necessary (and actually won't even work because of a chicken/egg problem of when the logs get created relative to running the triggers). Try running the command env in the Trigger Script and you'll see that Xcode actually sets environment variables with the results of the test, with some of the more notable ones being:

XCS_BOT_NAME=My New Bot
XCS_WARNING_CHANGE=0
XCS_INTEGRATION_RESULT=succeeded
XCS_TEST_FAILURE_COUNT=0
XCS_TEST_FAILURE_CHANGE=0
XCS_ERROR_COUNT=0
XCS_ANALYZER_WARNING_COUNT=0
XCS_TESTS_CHANGE=0
XPC_SERVICE_NAME=0
XCS_ERROR_CHANGE=0
XCS_WARNING_COUNT=0
XCS_TESTS_COUNT=3
XCS_INTEGRATION_NUMBER=1