How do I specify a separate maven goal for running (Cucumber) acceptance tests?

Question

I have the following project structure:

MyProject
   --src
   --test
      --acceptance
         --step_definitions
         --features
      --unit

I would like to be able to run my cucumber tests (in test/acceptance) separately in Maven from the unit tests declared in test/unit, so that they can be run in different CI build plans etc. I am using cucumber-junit so the 'runners' for each acceptance test are written with JUnit.

Is this possible?

Answer 1

Is this possible?

Yes, it is possible. I believe you should separate your unit from the acceptance/integration tests having:

Slightly modified folders structure for both of these, placing your integration test files in the standard location of src/it:

MyProject/

• src/main/java/ (SUT)
• src/test/ (unit test code)
  • java/
  • resources/
• src/it/ (acceptance/integration tests)
  • java/ (steps definitions)
  • resources/ (feature files)

Moreover, by design, different Maven plugins are intended for unit and integration tests:

• for unit tests: maven-surefire-plugin
• for acceptance/integration tests: maven-failsafe-plugin

You must also bind execution of maven-failsafe-pulgin. To run the integration tests separately, you can define a new profile:

<profiles>
  <profile>
    <id>acceptance-tests</id>
    <build>
      <plugins>
        <plugin>
          <artifactId>maven-failsafe-plugin</artifactId>
          <version>2.12</version>
          <executions>
            <execution>
              <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>    
    </build>
  </profile>
</profiles>

You will also need to configure the plugin to search the src/it directory tree for test cases.

The acceptance tests can be run afterwards using:

mvn clean verify -Pacceptance-tests

For complete sample, I'd suggest you to follow http://www.weblogism.com/item/334/integration-tests-with-cucumber-jvm-selenium-and-maven

Answer 2

The other answer suggested modifying your folder structure to have shared folder for integration and acceptance tests, but you can have the original folder structure as well. Also you mentioned in the comment you want to keep all three (including not-mentioned integration tests) separate, which is possible, though hackish.

Since you seem to have test/unit for unit tests and test/acceptance for acceptance test, I'm assuming test/integration for integration tests.

<profiles>
    <profile>
        <id>acceptance-test</id>
        <build>
            <plugins>
                <plugin>
                    <!-- to run directly: mvn failsafe:integration-test -P acceptance-test -->
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <version>2.12</version>
                    <configuration>
                        <testSourceDirectory>test/acceptance</testSourceDirectory>
                        <includes>
                            <include>**/*Acceptance.java</include>
                        </includes>
                        <excludes>
                            <exclude>**/*IT.java</exclude>
                            <exclude>**/*Test.java</exclude>
                        </excludes>
                    </configuration>
                    <executions>
                        <execution>
                            <goals>
                                <goal>integration-test</goal>
                                <goal>verify</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>
<build>
    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.9.1</version>
            <executions>
                <execution>
                    <id>add-test-source</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>add-test-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>test/unit</source>
                            <source>test/integration</source>
                            <source>test/acceptance</source>
                        </sources>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.12</version>
            <configuration>
                <testSourceDirectory>test/unit</testSourceDirectory>
            </configuration>
        </plugin>
        <plugin>
            <!-- to run directly: mvn failsafe:integration-test -->
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <testSourceDirectory>test/integration</testSourceDirectory>
            </configuration>
            <!-- execution below can be used, if tests are needed on 
                mvn:integration-test -->
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

Note though that separation only applies for sources: compiled files will all go to the same folder, and AFAIK that's something you can't change. This means you need to have naming strategy for your tests to separate them from each other.