Cuke4Nuke or SpecFlow?

Question

I am trying to decide if I should use Cuke4Nuke or SpecFlow. What are the pro/cons of each? Opinions on which is better and why.

Thanks!

Answer 1

(I might be biased because I am involved with SpecFlow, but here my thoughts...)

Cuke4Nuke is very close to Cucumber. This promises a lot of advantages:

• Compatibility
• Getting new features from Cucumber when Cucumber evolves (at least in theory, but language support is an example for this)
• Being a real part of the Cucumber community and the Cucumber ecosystem

However this comes also with some potential disadvantages:

• Ruby is a necessity
• Since more infrastructure (Ruby, Wire-Protocol, command-line integration...) is involved, the complexity of the whole solution rises, and chances that something in the chain is failing rise
• Debugging is possible but a bit of a hassle
• Running scenarios on the dos-commandline is just plain ugly, and I still have problems with some characters (German Umlaute). The solutions from Cucumber did not work for cuke4nuke in my case.
• Integration with your continuous build is something you have to work out for yourself

SpecFlow is a separate project from Cucumber. It tries to be as close to Cucumber as possible, but there are and will be gaps. There are plans to use the same parser as Cucumber, to improve compatibility on the language level.

SpecFlow tries to offer the following advantages:

• A pure .NET solution (so no installation of Ruby is necessary and Ruby is not involved at runtime)
• There is a basic integration with VisualStudio (and there are plans to evolve this)
• Scenarios are basically UnitTests and can be run with your existing infrastructure (NUnit.Runners, ReSharper, VisualStudio MSTest Integration ...)
• Scenarios and steps are easily debuggable out of VisualStudio (just set a breakpoint)
• Integration in your continuous build should be a breeze, since the infrastructure to run unit-tests is most certainly there already

As disadvantages of SpecFlow I see currently:

• It does not support as many languages as Cucumber
• Currently there is a "code generation" step involved. This is transparent when using VisualStudio, and there is a commandline to do this without VisualStudio, but a lot of people do not like code-generation.
• Currently there is no explicit commandline runner for SpecFlow. However you can use your unit-test commandline runner.
• SpecFlow depends on a Unit-Test framework, and currently only NUnit and MSTest is supported
• Reporting in SpecFlow is not very sophisticated yet. Cucumber does offer more options, however I don't know if they are all available in cuke4nuke...

Answer 2

jbandi gives a good summary. I answer the question in much the same way (with the opposite disclaimer for bias, of course).

The goal for Cuke4Nuke is full Cucumber compatibility in .NET while duplicating as little Cucumber code as possible. Therefore, some of the trade-offs you highlighted—e.g. the Ruby dependency—are inherent to the tool. Others, like bugs in language and formatter support and limited debugging support, are temporary issues and will go away with future versions.

I've run into a few issues where Cuke4Nuke doesn't work quite like Cucumber. But as I work primarily in English, I don't see the language-related issues in my normal work. I'd welcome steps to reproduce any of these issues so I can fix them. (Please post to them the Cuke4Nuke issues list, not here.)

Answer 3

Another heavily biased opinion: Try StoryQ :)

StoryQ tests are actually code, so you get much better refactoring / IDE support, and it embeds within your existing unit test runner, so CI is a breeze.

It's probably a matter of preference whether you'd rather check in plain text features or compilable code. But for us we found that it was really nice to be able to rename narrative methods and have all the stories update themselves.

There is actually a GUI provided that will convert plain text scenarios into StoryQ code for you, if you've already got an investment in plaintext scenarios or if you'd like to give the keyboard to your business people. It's even got a simple form of intellisense!

Give it a go if you want an ultra-lightweight entry point into BDD :)

Answer 4

Another biased response: StorEvil eats all other .NET BDD tools.

Advantages: StorEvil has its own command-line runner, has nice reporting (using the Spark view engine), and has the best plaintext->C# translation and execution engine.

Also, it has 100% more Evil than any other solution.

Disadvantages: StorEvil does not fully support other human languages (except English). StorEvil's Visual Studio integration is not yet as nice as the other tools. StorEvil will drink all the beer in the fridge if you don't keep an eye on it.

Answer 5

I understand from Richard that he intends to discontinue Cuke4Nuke and is supporting moving some of the Cuk4Nuke features into SpecFlow. So the clear answer now is SpecFlow.

Answer 6

I started with Cuke4Nuke but have since defected to SpecFlow (sorry Richard ;-)

The main reasons for me to make that transition were:

• SpecFlow has nice VS2010 integration for syntax highlighting of features. There is a Cuke4VS project that offers similar but it doesn't haven't have VS2010 support (or not when I last looked anyway, which was fairly recent)
• I found debugging tests in SpecFlow to be easier (don't ask me to elaborate, it just seemed that way... ;-)
• Cuke4Nuke needed Ruby. I was OK with that but most of the C# devs I know get spooked by any non-MS products, Ruby in particular.

There are some issues with Specflow/things I like better in the Cucumber/Cuke4Nuke world:

• Specflow's documentation is quite 'lite' -- you will have to be prepared to work hard to gather info from Cucumber sources and intuit a bit how they apply to Specflow. That said myself and few others have designs on beefing up the documentation so that may improve over the next few months.
• I much prefer the experience of running Cucumber/Cuke4Nuke tests on the command line with their output of the scenarios color coded according to status (I know somebody above sees this as a negative so I guess it depends whether you're a command line kind of guy...)
• The Cucumber community is bigger and there's more activity it seems which (possibly) translates to more people out there to help you.

All in all both have potential to improve the way we write software.