Unable to get JSCover and PhantomJS to run Jasmine test on Cloudbees

Question

I am currently trying to run JSCover in web server mode to determine the coverage of my Jasmine tests that are executed in the PhantomJS headless browser. I am also using grunt+nodejs to kick off the tests.

The code I use in my gruntfile to start the JSCover server and execute phantomJS is:

// Start JSCover Server
var childProcess = require('child_process'),
var JSCOVER_PORT = "43287";
var JAVA_HOME = process.env.JAVA_HOME;

var jsCoverChildArgs = [
        "-jar", "src/js/test/tools/JSCover-all.jar",
        "-ws",
        "--branch",
        "--port="+JSCOVER_PORT,
        "--document-root=./",
        "--report-dir=target/",
        "--no-instrument=src/js/lib/",
        "--no-instrument=src/js/test/",
        "--no-instrument=src/js/test/lib/"
    ];

var jsCoverProc = childProcess.spawn(JAVA_HOME + "/bin/java", jsCoverChildArgs);

// Start PhantomJS
var phantomjs = require('phantomjs'),
var binPath = phantomjs.path,

var childArgs = [
        'src/js/test/lib/phantomjs_jasminexml_runner.js',
        'http://localhost:'+JSCOVER_PORT+'/src/js/test/SpecRunner.html',
        'target/surefire-reports'
    ];
runner = childProcess.execFile(binPath, childArgs);

runner.on('exit', function (code) {
    // Tests have finished, so clean up the process
    var success = (code === 0) ? true : false;
    jsCoverProc.kill(); // kill the JSCover server now that we are done with it

    done(success);
}); 

However, when I run the web server on a Jenkins node in cloudbees and then run phantomjs against it, I get one of the following errors:

• Some tests start to run, but then the process fails:

  A spec : should be able to have a mock lo-dash ... 
  Warning: Task "test" failed. Use --force to continue.
  
  Aborted due to warnings.
  Build step 'Execute shell' marked build as failure
  Recording test results
  Finished: FAILURE
  

• PhantomJS is unable to access the JSCover server:

  Running "test" task
  phantomjs> Could not load 'http://127.0.0.1:43287/src/js/test/SpecRunner.html'.
  Warning: Task "test" failed. Use --force to continue.
  

For the second error, I have tried to use different ports and hostnames that I set (e.g. 127.0.0.1 or localhost for hostnames, and 4327, 43287, etc. for ports). The ports are not being dynamically set at build time - I have them hardcoded in my grunt script.

Any thoughts on why the errors above might be occurring or why I am having issues running and accessing the JSCover server on a Cloudbees Jenkins node (but never on my local machine)?

Answer 1

So when you execute JSCover with any process, it takes time to be up. If we expect it to be up earlier that it is, the errors are bound to come.

Quoting from the great article: http://blog.johnryding.com/post/46757192364/javascript-code-coverage-with-phantomjs-jasmine-and

Now that I had a code coverage tool that met all of my requirements, the last part was to get this code to run as part of our Jenkins build (which utilizes a grunt script). This was easy to get running, but I encountered two errors that consistently broke my builds:

1. Sometimes phantomJS would fail to connect to the JSCover server
2. Sometimes phantomJS would connect to the server, but then give up executing my tests at a random point during the run.

These were really weird issues that only occurred on my team’s Jenkins nodes and were hard to diagnose - even though they turned out to be simple fixes.

For issue 1, that error was the result of my grunt script not waiting for JSCover to start before I executed phantomJS.

For the second issue, it turns out that my team was using a special jasmine test runner to help with producing XML files after tests completed. The problem with this file was that it had a function that waited for Jasmine to complete its execution, but utilized an extremely short timeout before it gave up running the tests. This was a problem with Jenkins + JSCover because it took a longer time for the tests to load and run now that they had to be loaded from a web server instead of straight from the file system. Fortunately, this fix was as easy as increasing the timeout.

Answer 2

I would say that you need to wait for a while after spawning JSCover - in the past I have done things with webdriver when I have spawned, and then waited for it to be available (ideally you can look for a response and sleep, repeat, until the spawned process is ready).

Ie look for a valid http reponse from 127.0.0.1:43287 before continuing (whatever "valid" means that the server is up).