Commercial BDD tools for Java platform

Question

I have been experimenting with several acceptance test frameworks for Java during the last few weeks (e.g. Fitnesse, JBehave).

Before deciding which one to use I tried to find out whether there are any commercial alternatives but did not really find any. Any pointers appreciated.

Our requirements are (incomplete and unsorted list):

• Use of given/when/then-style texts AND of tables
• Integration for standard libraries, such as JUnit, Mockito
• IDE-integration/tooling
• Good reporting (readability for business analysts, diagnosing for developers)

Answer 1

For commercial alternatives:

• Thoughtworks' "Twist"
• Paul Gerrard's "Business Story Manager", which also plays into some of the higher-level BDD patterns, identifying stakeholders and tracking relationship of scenarios to larger-scale requirements

Those are the only two I know about. Neither have large followings in the BDD community. I believe the two most-used tools are JBehave and Cucumber (which as @kazakovs points out is also available for the JVM). These help to capture English-language scenarios which would be readable by both developers and business analysts. I've also used (the .NET equivalent of) Fitnesse with Slim behind it; a quick bit of CSS makes it look like the others, while allowing for tables as well.

However, most BDD tools are designed for full-stack, system-wide scenarios, usually with nothing mocked out, or possibly with third-party libraries replaced by a custom framework. Mocking libraries don't really have the set-up to work at this scale.

If you're looking to integrate Mockito and JUnit, it sounds as if you might be trying to do BDD at a smaller scale, with classes or small groups of classes. This is how BDD originally started and is perfectly valid.

Since BDD's inception, tools like JUnit have improved, and Mockito didn't exist! So BDD tools are no longer particularly necessary. For class level code, I would be happy to simply use comments with Given, When and Then in place, like this (this is C# but Java would be similar).

As an alternative at a higher level, you can always knock up a small DSL. This didn't take me very long, and it's readable, even for non-technical business people. It runs using JUnit, and you could put Mockito in there if you wanted (but I still suspect you're mixing concerns if you want to do this).

Another benefit of the DSL is that it's a quick, cheap way to get started, with none of the set-up overhead of Cucumber or JBehave; but the steps produced are easily ported to something like JBehave later on (you just move the code to JBehave's regex steps, and I'd be surprised if you couldn't do something similar with Twist). Since you're uncertain about your requirements, I would recommend this route as it will help you to get started and get more information about what you do need, very cheaply.

Answer 2

Take a look at the Cucumber-JVM But I'm not sure if it is commercial.

Answer 3

I've had a great success with Spock Framework if you can use Groovy. You can use it along with Geb to do ui testing as well. We have later switched from using Geb to Twist as it is cheaper for the great functionality it provides. We still use Spock for all service tests (WebService, REST etc). Read my another related answer here Demonstration using Spock

Answer 4

I would propose to have a look at Concordion (http://www.concordion.org) – yes, I have to admit that it is an open source tool and not a commercial one. Nevertheless, it could be a good fit based on the list of your requirements:

• As Concordion specifications are written in plain English, you can use given/when/then phrases as well as tables. In our projects we go even a step further and write our documentation based on the Concordion technology. As a result, you receive a living documentation system, where the automation checks that your description of the expected behaviour is in sync with the actual application.

• Concordion uses JUnit to run the tests / active specifications. In the fixture classes you can use any framework you prefer, like Mockito. I used myself the combination of Concordion and Mockito successfully in my projects.

• The IDE integration of Concordion is nice, as it is based on JUnit. Thus, you can use Concordion wherever you have JUnit support. I have been using the Eclipse IDE, where you can run and debug Concordion tests by calling “Run as JUnit test” from the context menu.

• Concordion provides HTML reports that are based on your specifications. Thus, your business people can easily see what went wrong by looking at the specified description and read the reported output the system provided. The differences are visualized by Concordion within HTML documents. Additionally, as Concordion tests are run by JUnit you receive all the reporting support you are probably used to by JUnit. You can easily debug Concordion tests, e.g. in the Eclipse IDE, like any other JUnit tests.