Running single test class or group with Surefire and TestNG

Question

I want to run single test class from command line using Maven and TestNG

Things that doesn't work:

mvn -Dtest=ClassName test

I have defined groups in pom.xml, and this class isn't in one of those groups. So it got excluded on those grounds.

mvn -Dgroups=skipped-group test
mvn -Dsurefire.groups=skipped-group test

when config is

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.1</version>
  <configuration>
    <groups>functest</groups>
  </configuration>
</plugin>

Parameters work fine in there are no groups defined in pom.xml.

Similarly, when surefire is configured with

<configuration>
  <includes>
    <include>**/*UnitTest.java</include>
  </includes>
</configuration> 

I can add another test with -Dtest parameter, but cannot add group. In any combination, I can narrow down tests to be executed with groups, but not expand them.

What's wrong with my configuration? Is there a way to run a single test or group outside of those defined in pom.xml?

Tried on Ubuntu 10.04 with Maven 2.2.1, TestNG 5.14.6 and Surefire 2.7.1

Answer 1

As I've explained in question, any mention of groups either in pom.xml or on command line resulted in reduction of executed tests count. Only way I've managed to avoid this is by using mavens profiles like this:

<profiles>
    <profile>
        <id>test-slow</id>
        <build>
            <plugins>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <groups>slow</groups>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

and then running tests with

mvn -P test-slow test

Answer 2

I didn't test with TestNG 5.12.1 but I can say that running a single test using the test parameter and tests from groups using the groups parameter works with TestNG 5.14.2 (and surefire 2.6) (groups doesn't work in TestNG 5.14)

Here is the pom.xml I'm using:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.stackoverflow</groupId>
  <artifactId>Q4159948</artifactId>
  <version>1.0-SNAPSHOT</version>
  <name>Q4159948</name>
  <url>http://maven.apache.org</url>
  <properties>
    <maven.compiler.source>1.6</maven.compiler.source>
    <maven.compiler.target>1.6</maven.compiler.target>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.testng</groupId>
      <artifactId>testng</artifactId>
      <version>5.14.2</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.6</version>
        <configuration/>
      </plugin>
    </plugins>
  </build>
</project>

With a simple AppTest as follow:

import org.testng.annotations.*;

public class AppTest {

 @BeforeClass
 public void setUp() {
   // code that will be invoked when this test is instantiated
 }

 @Test(groups = { "fast" })
 public void aFastTest() {
   System.out.println("Fast test");
 }

 @Test(groups = { "slow" })
 public void aSlowTest() {
    System.out.println("Slow test");
 }

}

Both

$ mvn test -Dtest=AppTest

and

$ mvn test -Dgroups=slow

produce the expected result.

Answer 3

I would suggest to try something like

mvn test -Dincludes=rs/magrathea/TestClassName

although I haven't tested this myself.

Answer 4

In order to run a single test you need the following from official documentation

mvn -Dtest=MyFirstTest test

or

mvn -Dtest=MyFirstTest,MySecondTest test

This is tested (and working) on maven 3.

Then you can avoid using the profiles. I had the same problem as I needed to run load test in isolation and using profiler in parallel to get the real figures.

Note: Not sure why but make sure that the switches come before the phase i.e. "-Dtest=MyFirstTest" before "test" otherwise it is not working (Mac OSX)