How can I get Test Result during executing WebDriver tests in Jenkins?
-
13-11-2019 - |
質問
I'm using the Play! WebDrive Module and the jobs run okay but when tests fail the Jenkins job shows 'All tests passed'. I'm trying to wire in test results but I keep running into troubles.
I tried using the SeleniumHQ plugin for Jenkins but that results in the error shown below. I use Firefox so I had the test results location set to {app}/test-result/FirefoxDriver/*.html
Publishing Selenium report...
ERROR: No Test Report Found
Build step 'Publish Selenium Report' changed build result to FAILURE
Finished: FAILURE
So I guess the output isn't really Selenium. I tried pulling in the Play! Plugin and that didn't give any good options. I did try out the Play! auto-test reports post-build step but that only resulted in showing the application log.
Finally, I tried using the Publish testing tools result report post-build step. This one is particularly frustrating because docs are so awful and Jenkins doesn't provide any debugging / trouble-shooting info that I can find.
Custom Tool Pattern: {app}/test-result/FirefoxDriver/*.html
Custom stylesheet: scripts/webdrive2junit.xsl
The XSL that I set up is below and then the error that I consistently get is below that. Is anyone getting test results from webdrive:test into the Jenkins job?
<?xml version="1.0" encoding="UTF-8"?>
<!--
- Hopefully we won't use this for long. I put it together to get some information
- out of the webdrive:test results while I look for a better solution.
-->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="child::body">
<xsl:variable name="test" select="normalize-space(descendant::div/h1)"/>
<testsuite>
<xsl:attribute name="name">
<xsl:value-of select="$test"/>
</xsl:attribute>
<testcase>
<xsl:attribute name="classname">
<xsl:value-of select="$test"/>
</xsl:attribute>
<xsl:attribute name="name">
<xsl:value-of select="descendant::div/div/table/thead/tr/th"/>
</xsl:attribute>
<xsl:if test="descendant::tr[@class=' status_failed']">
<error />
</xsl:if>
</testcase>
</testsuite>
</xsl:template>
</xsl:stylesheet>
Here's the error that I get in the job log.
[xUnit] [INFO] - [Custom Tool] - 1 test report file(s) were found with the pattern 'src/test-result/FirefoxDriver/*.html' relative to '/opt/ci/hudson/workspace/enterprise_another-play-test' for the testing framework 'Custom Tool'.
[xUnit] [ERROR] - Conversion error Error to convert - A file not found
ERROR: Publisher org.jenkinsci.plugins.xunit.XUnitPublisher aborted due to exception
hudson.util.IOException2: remote file operation failed: /opt/ci/hudson/workspace/enterprise_another-play-test at hudson.remoting.Channel@34b246:build-trunk-2
...
Caused by: com.thalesgroup.hudson.plugins.xunit.exception.XUnitException: Conversion error Error to convert - A file not found
at com.thalesgroup.hudson.plugins.xunit.service.XUnitConversionService.convert(XUnitConversionService.java:89)
at com.thalesgroup.hudson.plugins.xunit.service.XUnitTransformer.invoke(XUnitTransformer.java:135)
... 11 more
Caused by: com.thalesgroup.dtkit.util.converter.ConversionException: Error to convert - A file not found
at com.thalesgroup.dtkit.util.converter.ConversionService.convert(ConversionService.java:369)
at com.thalesgroup.dtkit.util.converter.ConversionService.convert(ConversionService.java:177)
at com.thalesgroup.dtkit.util.converter.ConversionService.convert(ConversionService.java:114)
at com.thalesgroup.dtkit.metrics.model.InputMetricXSL.convert(InputMetricXSL.java:196)
at com.thalesgroup.dtkit.metrics.model.InputMetricXSL.convert(InputMetricXSL.java:202)
at com.thalesgroup.hudson.plugins.xunit.service.XUnitConversionService.convertCustomInputMetric(XUnitConversionService.java:104)
at com.thalesgroup.hudson.plugins.xunit.service.XUnitConversionService.convert(XUnitConversionService.java:78)
... 12 more
Caused by: org.xml.sax.SAXParseException: The element type "meta" must be terminated by the matching end-tag "</meta>".
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(Unknown Source)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at com.thalesgroup.dtkit.util.converter.ConversionService.convert(ConversionService.java:324)
... 18 more
Finished: FAILURE
解決
My current work-around is below. Any improvements would be appreciated. Since the Play plugin doesn't work well for me (because of the webdrive module is appears) my build steps are bash scripts.
# work-around for the XUnit Plugin trouble; feel free to Tidy up
find src/test-result/FirefoxDriver -name "*.html" -print0 | xargs -r0 -n 1 bash -c '
testname=$(basename ${0%.html.*.html});
xsltproc --html -o src/test-result/FirefoxDriver/TEST-$testname.xml scripts/webdrive2junit.xsl $0
'
I looked at the code for the XUnit Plugin to Jenkins (which I believe is behind the Publish testing tools result report post-build step) and I realize that it is not doing what I thought.