Test harness for lab setup

Question

We are new to test automation, and finally we finally automated our setup (black box testing). A brief overview of our setup:

Each setup consists of 2 linux PCs and 1 windows PC - the PCs communicate to one another via an embedded board.

Before we used to manually run test cases from the linux machine - this would require both typing in the Linux machine and also some operations in the windows PC. Now instead we have written a C wrapper from which you can trigger any number of test cases and thanks to AutoIT the windows PC operations are now automated as well.

Now we have multiple such setups - I want to have a central test controller that,

1. Given a set of test cases (and the corresponding executable to be run on the embedded board) can distribute and trigger it in parallel across setups
2. During overnight tests it can keep track of which test cases have been executed and which ones are pending
3. Quarantine test cases
4. Continuous integration - we use CVS
5. etc etc

Basically a powerful test harness software running on a PC - this PC is connected to all the setups using a router.

Any suggestions from open source (free) projects for such a software, more than having all the features mentioned above, i want something that does most of it, I can code and add additional functionalities as need

I tried browsing online and seem to find some but they all seem to be for testing websites, not sure if it would suit my use case. Would really appreciate inputs in this regard.

Thanks

Answer 1

ok i am going to ignore most of the stuff that you described which looks like you have automated already, and correct me if i am wrong but i think you are looking for is controller for all these automated jobs.

I would say that Jenkins CI is the ideal solution for you.

In jenkins everything is controlled by a master machine, this machine then in turn controls slaves (being your other pcs or linux machines) via java. so this gives you overview of the system.

you can then create jobs and ristrict where they can run, these jos can pretty much do anything. Including taking parameters for what to run, you can also create matrix configuration jobs which allow for one setup to be run simlutansily on however many slaves you need.

You can set this process or timer, or trigger , build externaly or internaly or etc etc ...

Also I am pretty sure Jenkins has some dedicated plugins for working with CVS some of which are built into the OOTB setup.

Jenkins is the way to go here .