ScalaTest runner + FunSuite + tag: can't get `-n` and `-l` to include/exclude tag?

Question

The ScalaTest runner is supposed to support including or excluding tests by tag, but I can't get it to work. What am I doing wrong? Details follow.

in an otherwise empty directory, given build.sbt:

libraryDependencies += "org.scalatest" %% "scalatest" % "2.1.0" % "test"

and src/test/scala/Test.scala:

class Test extends org.scalatest.FunSuite {
  test("fast") {
    assertResult(4)(2 + 2) }
  test("slow", org.scalatest.tagobjects.Slow) {
    assertResult(40)(20 + 20) }
}

then:

% sbt
[info] Set current project to bug (in build file:/Users/tisue/bug/)
> show sbtVersion
[info] 0.13.1
> test
[info] Compiling 1 Scala source to /Users/tisue/bug/target/scala-2.10/test-classes...
[info] Test:
[info] - fast
[info] - slow
[info] Run completed in 192 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 3 s, completed Mar 15, 2014 12:25:06 PM

so far so good. but now:

> test-only Test -- -l org.scalatest.tagobjects.Slow
[info] Test:
[info] - fast
[info] - slow
[info] Run completed in 108 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 0 s, completed Mar 15, 2014 12:25:39 PM
> test-only Test -- -n org.scalatest.tagobjects.Slow
[info] Test:
[info] Run completed in 93 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[success] Total time: 0 s, completed Mar 15, 2014 12:25:42 PM

using -l to exclude the tag didn't exclude anything, and using -n to include the tag didn't include anything.

The problem isn't specific to sbt/ScalaTest integration, because it also fails if I run the runner directly:

> test:run-main org.scalatest.tools.Runner -o -R target/scala-2.10/test-classes -l org.scalatest.tagobjects.Slow
[info] Running org.scalatest.tools.Runner -o -R target/scala-2.10/test-classes -l org.scalatest.tagobjects.Slow
Discovery starting.
Discovery completed in 52 milliseconds.
Run starting. Expected test count is: 2
Test:
- fast
- slow
Run completed in 96 milliseconds.
Total number of tests run: 2
Suites: completed 2, aborted 0
Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
All tests passed.

Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "ScalaTest-main"
[success] Total time: 0 s, completed Mar 15, 2014 12:30:14 PM
> test:run-main org.scalatest.tools.Runner -o -R target/scala-2.10/test-classes -n org.scalatest.tagobjects.Slow
[info] Running org.scalatest.tools.Runner -o -R target/scala-2.10/test-classes -n org.scalatest.tagobjects.Slow
Discovery starting.
Discovery completed in 51 milliseconds.
Run starting. Expected test count is: 0
Test:
Run completed in 86 milliseconds.
Total number of tests run: 0
Suites: completed 2, aborted 0
Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
No tests were executed.

Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "ScalaTest-main"
[success] Total time: 0 s, completed Mar 15, 2014 12:30:18 PM

What am I doing wrong?

Answer 1

You need to use "org.scalatest.tags.Slow" as the tag name, as described in the ScalaDoc for Slow.

Unfortunately it is not possible to have an annotation and object have the same fully qualified name, so we use the fully qualified name of the annotation as the tag name.

Answer 2

Working example:

As mentioned in http://doc.scalatest.org/2.1.0/index.html#org.scalatest.FunSuite in "Tagging tests" you must pass an object which extends org.scalatest.Tag. For instance:

import org.scalatest.FlatSpec
import org.scalatest.Tag
import org.scalatest.tags.Slow

object Other extends Tag("com.mycompany.groups.Other")

class Test extends org.scalatest.FunSuite {
  test("fast") {
    assertResult(4)(2 + 2) }

  test("slow", org.scalatest.tagobjects.Slow ) {
    assertResult(40)(20 + 20) }

  test("other", Other){
    assertResult(42)(21 + 21)   }
}