Fitnesse - which CI automation is better for running Fitnesse tests in parallel?

Question

we're running our CI loop with CruiseControl - and have more than a thousand of FitNesse/FitLibrary tests. It takes awhile, to say the least. Some of the tests have to run sequentially, but some can run in parallel (most of them). Does anyone have a good experience configuring a complex FitNess CI environment w/CruiseControl? How? Or - let me rephrase it - which test automation control server is better in this respect (with FitNesse)? Thanks!

Answer 1

The trick to running FitNesse servers in parallel is mostly about keeping all of the ports separate for each of the runs. So each FitNesse server needs to be started with a different -p argument. As to the FitLibrary part, well if you do spin up something that your FitLibrary fixtures talk to over a port, make sure they use different ones. If you we're using SliM, then the would be at lease one more port (SLIM_PORT) you would need to make different foe each parallel thread.

The other thing that you might want to be careful about is launching all of the tests from the same source directory. FitNesse itself shouldn't have a problem, but if your fixtures need to work with any physical files, those can sometimes cause trouble. You can also run into conflicts when updating the folder if one job is testing when another job kicks off an svn update or similar operation in Git/CVS.

The only other gotcha I know about is whether any other libraries or the system you are testing can have trouble with multiple instances running at once.

We run multiple execution threads under Hudson/Jenkins all of the time. Each job runs in a separate execution thread. Typically with a separate test system to make sure there tests that interact with the database don't come in conflict.

If CruiseControl can run multiple builds in parallel, it probably can run multiple tests too.