How can I get Scalatest to give me one stats summary when testing in a loop

Question

I am trying to run a set of tests on multiple XML documents. I want to get a list of product IDs from a config file, and then run the same set of tests on every document. However, when I do this, I cannot get a single final summary of the test stats.

Sample code is below:

import org.scalatest._
import org.scalatest.matchers.ShouldMatchers._
import scala.xml._
import dispatch._

class xyzSpec(webcli: Http, productId: String) extends FeatureSpec with GivenWhenThen with ShouldMatchers {

    feature("We get up to date xyz data from xyzsystem with correct blahblah info") {
    info("As a programmer")
    info("I want to lookup a product in xyzsystem")
    info("So that I can check the date updated and blahblah info")
    scenario("We have an up to date product with correct blahblah info") {
        given("Product " + productId)
            // code to get product XML doc
        when("when we request the db record")
            // code to get crosscheck data from SQL db
        then("we can get the product record") 
            // code to compare date updated
        and("date updated in the XML matches the SQL db")   

   }
  }

}

val h = new Http

val TestConfXml = h(qaz <> identity)
ProdIdsXml \\ "product" foreach {  (product) =>
    val productId = (product \ "@id").text
    new xyzSpec(h, productId).execute(stats=true)        

}

The third last line has a foreach which invokes the test runner multiple times. I know that I can nest test objects (or is that test classes) but I cannot see how to do this dynamically at runtime, when the test class constructor takes parameters.

What am I missing?

Answer 1

Rather than call execute on each suite you create, just collect them then return them from nestedSuites of a suite. And call execute on that outer suite.

scala> import org.scalatest._
import org.scalatest._

scala> class NumSuite(num: Int) extends Suite {
     |   override def suiteName = "NumSuite(" + num + ")"
     | }
defined class NumSuite

scala> val mySuites = for (i <- 1 to 5) yield new NumSuite(i)
mySuites: scala.collection.immutable.IndexedSeq[NumSuite] = Vector(NumSuite@3dc1902d,     NumSuite@6ee09a07, NumSuite@5ba07a6f, NumSuite@4c63c68, NumSuite@72a7d24a)

scala> stats.run(Suites(mySuites: _*))
Run starting. Expected test count is: 0
Suites:
NumSuite(1):
NumSuite(2):
NumSuite(3):
NumSuite(4):
NumSuite(5):
Run completed in 16 milliseconds.
Total number of tests run: 0
Suites: completed 6, aborted 0
Tests: succeeded 0, failed 0, ignored 0, pending 0
All tests passed.

Voila!