How compatible are rspec and heckle?

Question

I'm currently using test/unit, and I'm considering using rspec. However, I've noticed that rspec currently doesn't support heckle in ruby 1.9.1, and doesn't support passing any parameters to heckle apart from the target module/class/method.

Are there any other current problems with using heckle and rspec, or do they work well together apart from those two issues?

Also, if I stay with test/unit for the moment, should I consider using it with cucumber?

Edit: The 1.9.1 issue isn't rspec's fault - heckle can't run on ruby 1.9.1 because parsetree can't run on ruby 1.9.1.

Answer 1

The last time I checked, Heckle+RSpec didn't work very well, and I don't think many people are using them together.

Back in the spring 2008 I looked into this and noticed that RSpec and Heckle hadn't been working together at all for some time. I made a patch and it was accepted into RSpec 1.1.14. Even after those changes, I didn't find Heckle very usable. If I remember correctly the problem is that Heckle itself crashes with non-trivial programs.

It is really a pity that the situation is what it is. If you want to help, you may want to look at the boo-hiss project: http://github.com/halorgium/boo_hiss/tree/master .

Update: After a two-year hiatus, a new version of Heckle (Heckle 1.4.2) was released last week. Maybe this has fixed the problems I experienced.

Answer 2

I am currently running heckle 1.4.2 and rspec 1.2.4 to test Addressable. I recommend using the spec command directly to heckle anything rather than setting it up through a rake task. Heckling is something I typically do just before a release, rather than during regular development because it's so time-intensive. And when you're trying to track down surviving mutations, you're likely to want to go method-by-method.

Older versions of RSpec were much chattier — dumping the full spec results for all failing specs — but I sent through a patch that fixed that. Any of the recent versions don't have that problem, and heckling should work just fine.

For example:

spec spec/**/*_spec.rb --heckle Addressable::URI#normalize

Output:

**********************************************************************
***  Addressable::URI#normalize loaded with 25 possible mutations
**********************************************************************

25 mutations remaining...
24 mutations remaining...
23 mutations remaining...
22 mutations remaining...
21 mutations remaining...
20 mutations remaining...
19 mutations remaining...
18 mutations remaining...
17 mutations remaining...
16 mutations remaining...
15 mutations remaining...
14 mutations remaining...
13 mutations remaining...
12 mutations remaining...
11 mutations remaining...
10 mutations remaining...
9 mutations remaining...
8 mutations remaining...
7 mutations remaining...
6 mutations remaining...
5 mutations remaining...
4 mutations remaining...
3 mutations remaining...
2 mutations remaining...
1 mutations remaining...
No mutants survived. Cool!

(Results not typical.)