How can I use Jenkins to run my integration tests in parallel?

Question 1

I found this great article on how to do this, but it gives a way to do it in Groovy code. I pretty much followed these steps, but I haven't written the code to distribute the tests evenly by duration. But this is still a useful tool so I'll share it.

import junit.framework.JUnit4TestAdapter;
import junit.framework.TestSuite;
import org.junit.Ignore;
import org.junit.extensions.cpsuite.ClassesFinder;
import org.junit.extensions.cpsuite.ClasspathFinderFactory;
import org.junit.extensions.cpsuite.SuiteType;
import org.junit.runner.RunWith;
import org.junit.runners.AllTests;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

@RunWith(AllTests.class)
public class DistributedIntegrationTestRunner {

    private static Logger log = LoggerFactory.getLogger(DistributedIntegrationTestRunner.class);

    public static TestSuite suite() {
        TestSuite suite = new TestSuite();

        ClassesFinder classesFinder = new ClasspathFinderFactory().create(true,
                new String[]{".*IntegrationTest.*"},
                new SuiteType[]{SuiteType.TEST_CLASSES},
                new Class[]{Object.class},
                new Class[]{},
                "java.class.path");

        int nodeNumber = systemPropertyInteger("node.number", "0");
        int totalNodes = systemPropertyInteger("total.nodes", "1");

        List<Class<?>> allTestsSorted = getAllTestsSorted(classesFinder);
        allTestsSorted = filterIgnoredTests(allTestsSorted);
        List<Class<?>> myTests = getMyTests(allTestsSorted, nodeNumber, totalNodes);
        log.info("There are " + allTestsSorted.size() + " tests to choose from and I'm going to run " + myTests.size() + " of them.");
        for (Class<?> myTest : myTests) {
            log.info("I will run " + myTest.getName());
            suite.addTest(new JUnit4TestAdapter(myTest));
        }

        return suite;
    }

    private static int systemPropertyInteger(String propertyKey, String defaultValue) {
        String slaveNumberString = System.getProperty(propertyKey, defaultValue);
        return Integer.parseInt(slaveNumberString);
    }

    private static List<Class<?>> filterIgnoredTests(List<Class<?>> allTestsSorted) {
        ArrayList<Class<?>> filteredTests = new ArrayList<Class<?>>();
        for (Class<?> aTest : allTestsSorted) {
            if (aTest.getAnnotation(Ignore.class) == null) {
                filteredTests.add(aTest);
            }
        }
        return filteredTests;
    }

    /*
    TODO: make this algorithm less naive.  Sort each test by run duration as described here: http://blog.tradeshift.com/just-add-servers/
     */
    private static List<Class<?>> getAllTestsSorted(ClassesFinder classesFinder) {
        List<Class<?>> allTests = classesFinder.find();
        Collections.sort(allTests, new Comparator<Class<?>>() {
            @Override
            public int compare(Class<?> o1, Class<?> o2) {
                return o1.getSimpleName().compareTo(o2.getSimpleName());
            }
        });
        return allTests;
    }

    private static List<Class<?>> getMyTests(List<Class<?>> allTests, int nodeNumber, int totalNodes) {
        List<Class<?>> myTests = new ArrayList<Class<?>>();

        for (int i = 0; i < allTests.size(); i++) {
            Class<?> thisTest = allTests.get(i);
            if (i % totalNodes == nodeNumber) {
                myTests.add(thisTest);
            }
        }

        return myTests;
    }
}

The ClasspathFinderFactory is used to find all test classes that match the .*IntegrationTest pattern.

I make N jobs and they all run this Runner but they all use different values for the node.number system property, so each job runs a different set of tests. This is how the failsafe plugin looks:

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.12.4</version>
            <executions>
                <execution>
                    <id>integration-tests</id>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <includes>
                    <include>**/DistributedIntegrationTestRunner.java</include>
                </includes>
                <skipITs>${skipITs}</skipITs>
            </configuration>
        </plugin>

The ClasspathFinderFactory comes from

        <dependency>
            <groupId>cpsuite</groupId>
            <artifactId>cpsuite</artifactId>
            <version>1.2.5</version>
            <scope>test</scope>
        </dependency>

I think there should be some Jenkins plugin for this, but I haven't been able to find one. Something that's close is the Parallel Test Executor, but I don't think this does the same thing I need. It looks like it runs all the tests on a single job/server instead of multiple servers. It doesn't provide an obvious way to say, "run these tests here, and those tests there".

Question 2

I believe you already found a solution by now, but I'll leave a path for the others who'll open this page asking the same question:
Parallel test executor plugin:
"This plugin adds a new builder that lets you easily execute tests defined in a separate job in parallel. This is achieved by having Jenkins look at the test execution time of the last run, split tests into multiple units of roughly equal size, then execute them in parallel."
https://wiki.jenkins-ci.org/display/JENKINS/Parallel+Test+Executor+Plugin

Question 3

Yes, Parallel Test Executor is a cool plugin if you've got 2 slave or one slave with 8 executor because the this plugin based on "tests splitting" so e.g: you split your junit tests into 4 different array, these arrays will run on 4 different executor on that slave what you specified. I hope you got it :D, it depends on the number of executors on that slave where you want to run parallel testing or you should decrease split tests count to 2 from 4.