Automated Unit Testing with JavaScript

StackOverflow https://stackoverflow.com/questions/96066

Question

I'm trying to incorporate some JavaScript unit testing into my automated build process. Currently JSUnit works well with JUnit, but it seems to be abandonware and lacks good support for AJAX, debugging, and timeouts.

Has anyone had any luck automating (with ANT) a unit testing library such as YUI test, JQuery's QUnit, or jQUnit (http://code.google.com/p/jqunit/)?

Note: I use a custom built AJAX library, so the problem with Dojo's DOH is that it requires you to use their own AJAX function calls and event handlers to work with any AJAX unit testing.

Was it helpful?

Solution

There are many javascript unit test framework out there (jsUnit, scriptaculous, ...) but jsUnit is the only one I know that may be used with an automated build.

If you are doing 'true' unit test you should not need AJAX support. For example, if you are using an RPC ajax framework such as DWR, you can easily write a mock function :

   function mockFunction(someArg, callback) {
      var result = ...; // some treatments   
      setTimeout(

        function() { callback(result); }, 
        300  // some fake latency

      );
    }

And yes, JsUnit do handle timeouts : Simulating Time in jsUnit Tests

OTHER TIPS

I'm just about to start doing Javascript TDD on a new project I am working on. My current plan is to use qunit to do the unit testing. While developing the tests can be run by simply refreshing the test page in a browser.

For continuous integration (and ensuring the tests run in all browsers), I will use Selenium to automatically load the test harness in each browser, and read the result. These tests will be run on every checkin to source control.

I am also going to use JSCoverage to get code coverage analysis of the tests. This will also be automated with Selenium.

I'm currently in the middle of setting this up. I'll update this answer with more exact details once I have the setup hammered out.

Testing tools:

Im a big fan of js-test-driver

It works well in a CI environment and is able to capture actual browsers for cross-browser testing.

I recently read an article by Bruno using JsUnit and creating a JsMock framework on top of that... very interesting. I'm thinking of using his work to start unit testing my Javascript code.

Mock Javascript or How to unit test Javascript outside the Browser environment

I just got Hudson CI to run JasmineBDD (headless), at least for pure javascript unit testing.

(Hudson running Java via shell, running Envjs, running JasmineBDD.)

I haven't got it to play nice with a big library yet, though, like prototype.

Look into YUITest

I am in agreement that jsunit is kind of dying on the vine. We just finished up replacing it with YUI Test.

Similar to the example using qUnit, we are running the tests using Selenium. We are running this test independently from our other selenium tests simply because it does not have the dependencies that the normal UI regression tests have (e.g. deploying the app to a server).

To start out, we have a base javascript file that is included in all of our test html files. This handles setting up the YUI instance, the test runner, the YUI.Test.Suite object as well as the Test.Case. It has a methods that can be accessed via Selenium to run the test suite, check to see if the test runner is still running (results are not available until after it's done), and get the test results (we chose JSON format)

var yui_instance; //the YUI instance
var runner;  //The YAHOO.Test.Runner
var Assert; //an instance of YAHOO.Test.Assert to save coding
var testSuite; //The YAHOO.Test.Suite that will get run.

/**
 * Sets the required value for the name property on the given template, creates
 * and returns a new YUI Test.Case object.
 * 
 * @param template the template object containing all of the tests
 */
function setupTestCase(template) {
    template.name = "jsTestCase";
    var test_case = new yui_instance.Test.Case(template);
    return test_case;
}

/**
 * Sets up the test suite with a single test case using the given 
 * template.
 * 
 * @param template the template object containing all of the tests
 */
function setupTestSuite(template) {
    var test_case = setupTestCase(template);
    testSuite = new yui_instance.Test.Suite("Bond JS Test Suite");
    testSuite.add(test_case);
}

/**
 * Runs the YAHOO.Test.Suite
 */
function runTestSuite() {
    runner = yui_instance.Test.Runner;
    Assert = yui_instance.Assert;

    runner.clear();
    runner.add(testSuite);
    runner.run();
}

/**
 * Used to see if the YAHOO.Test.Runner is still running.  The
 * test results are not available until it is done running.
 */
function isRunning() {
    return runner.isRunning();
}

/**
 * Gets the results from the YAHOO.Test.Runner
 */
function getTestResults() {
    return runner.getResults(yui_instance.Test.Format.JSON);
}

As for the selenium side of things, we used a parameterized test. We run our tests in both IE and FireFox in the data method, parsing the test results into a list of Object arrays with each array containing the browser name, the test file name, the test name, the result (pass, fail or ignore) and the message.

The actual test just asserts the test result. If it is not equal to "pass" then it fails the test with the message returned from the YUI Test result.

    @Parameters
public static List<Object[]> data() throws Exception {
    yui_test_codebase = "file:///c://myapppath/yui/tests";

    List<Object[]> testResults = new ArrayList<Object[]>();

    pageNames = new ArrayList<String>();
    pageNames.add("yuiTest1.html");
    pageNames.add("yuiTest2.html");

    testResults.addAll(runJSTestsInBrowser(IE_NOPROXY));
    testResults.addAll(runJSTestsInBrowser(FIREFOX));
    return testResults;
}

/**
 * Creates a selenium instance for the given browser, and runs each
 * YUI Test page.
 * 
 * @param aBrowser
 * @return
 */
private static List<Object[]> runJSTestsInBrowser(Browser aBrowser) {
    String yui_test_codebase = "file:///c://myapppath/yui/tests/";
    String browser_bot = "this.browserbot.getCurrentWindow()"
    List<Object[]> testResults = new ArrayList<Object[]>();
    selenium = new DefaultSelenium(APPLICATION_SERVER, REMOTE_CONTROL_PORT, aBrowser.getCommand(), yui_test_codebase);
    try {
        selenium.start();

        /*
         * Run the test here
         */
        for (String page_name : pageNames) {
            selenium.open(yui_test_codebase + page_name);
            //Wait for the YAHOO instance to be available
            selenium.waitForCondition(browser_bot + ".yui_instance != undefined", "10000");
            selenium.getEval("dom=runYUITestSuite(" + browser_bot + ")");

            //Output from the tests is not available until 
            //the YAHOO.Test.Runner is done running the suite
            selenium.waitForCondition("!" + browser_bot + ".isRunning()", "10000");
            String output = selenium.getEval("dom=getYUITestResults(" + browser_bot + ")");

            JSONObject results = JSONObject.fromObject(output);
            JSONObject test_case = results.getJSONObject("jsTestCase");
            JSONArray testCasePropertyNames = test_case.names();
            Iterator itr = testCasePropertyNames.iterator();

            /*
             * From the output, build an array with the following:
             *  Test file
             *  Test name
             *  status (result)
             *  message
             */
            while(itr.hasNext()) {
                String name = (String)itr.next();
                if(name.startsWith("test")) {
                    JSONObject testResult = test_case.getJSONObject(name);
                    String test_name = testResult.getString("name");
                    String test_result = testResult.getString("result");
                    String test_message = testResult.getString("message");
                    Object[] testResultObject = {aBrowser.getCommand(), page_name, test_name, test_result, test_message};
                    testResults.add(testResultObject);
                }
            }

        }
    } finally {
        //if an exception is thrown, this will guarantee that the selenium instance
        //is shut down properly
        selenium.stop();
        selenium = null;
    }
    return testResults;
}
/**
 * Inspects each test result and fails if the testResult was not "pass"
 */
@Test
public void inspectTestResults() {
    if(!this.testResult.equalsIgnoreCase("pass")) {
        fail(String.format(MESSAGE_FORMAT, this.browser, this.pageName, this.testName, this.message));
    }
}

I hope this is helpful.

There's a new project that lets you run qunit tests in a Java environment (like ant) so you can fully integrate your client-side test suite with your other unit tests.

http://qunit-test-runner.googlecode.com

I've used it to unit test jQuery plugins, objx code, custom OO JavaScript and it works for everything without modification.

The project I'm working on uses Js-Test-Driver hosting Jasmine on Chrome 10 with Jasmine-JSTD-Adapter including making use of Code Coverage tests included in JS-Test-Driver. While there are some problems each time we change or update browsers on the CI environment the jasmine tests are running pretty smoothly with only minor issues with ansynchronous tests, but as far as I'm aware these can be worked around using Jasmine Clock but I haven't had a chance to patch them yet.

I've published a little library for verifying browser-dependent JavaScript tests without having to use a browser. It is a node.js module that uses zombie.js to load the test page and inspect the results. I've wrote about it on my blog. Here is what the automation looks like:

var browsertest = require('../browsertest.js').browsertest;

describe('browser tests', function () {

it('should properly report the result of a mocha test page', function (done) {
    browsertest({
        url: "file:///home/liam/work/browser-js-testing/tests.html",
        callback: function() { 
            done();
        }
    });
});

});

I looked on your question date and back then there were a few good JS testing lib/framework. Today you can find much more and in different focus like TDD, BDD, Assetion and with/without runners support.

There are Many players in this game like Mocha, Chai, QUnit, Jasmine, etc... You can find some more information in this blog about JS/Mobile/web testing...

Another JS testing framework that can be run with Ant is CrossCheck. There's an example of running CrossCheck via Ant in the build file for the project.

CrossCheck attempts, with limited success, to emulate a browser, including mock-style implementations of XMLHttpRequest and timeout/interval.

It does not currently handle loading javascript from a web page, though. You have to specify the javascript files that you want to load and test. If you keep all of your JS separated from your HTML, it might work for you.

I've written an Ant task which uses Phantom JS, a headless webkit browser, to run QUnit html test files within an Ant build process. It can also fail the build if any tests fail.

https://github.com/philmander/ant-jstestrunner

This is a good evaluation of several testing tools.

JavaScript unit test tools for TDD

I personally prefer https://code.google.com/p/js-test-driver/

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top