Is there a consensus about test frameworks for Ruby 1.9.x? [closed]

Question

A two-parter with a quick intro. Intro: I'm coming to Ruby from Perl, and I'm a bit lost among the test framework choices. I understand that there is probably no single, all-around best choice, but I can't even get a clear picture of the playing field.

So, first, MiniTest or Test::Unit? I just realized that 1.9.1 defaults to MiniTest now. I hadn't noticed since all the older Test::Unit tutorials still work fine. If you require test/unit, you actually get MiniTest as a Test::Unit emulator (!). My initial thought was to go with the default choice of MiniTest (once I knew it was there), but even very recent books don't seem interested in teaching it. Ruby Best Practices says "There are significant differences" between the two, but he chooses not to deal with them. He also runs all his tests via Test::Unit (plus some custom add-ons). The new Pickaxe also has very little about MiniTest per se, and in the chapter on testing they also use the Test::Unit emulation version of MiniTest. To clarify my first question then: is MiniTest a poor first option? Should I default to using the Test::Unit emulator?

Second, beyond the built-in options, there are Rspec, Cucumber & company for behavior driven tests. Is the community tilting strongly towards this style and this framework? Should I just start with Rspec?

I will play a bit with all three, but I'm curious to hear what kind of consensus (if any) is forming around testing in Ruby 1.9.

(A quick follow-up, as a reminder to myself and for anyone interested. See this link for a useful overview of the different invocations for Test::Unit and MiniTest in the various Rubies 1.8 to 1.9.1.)

Answer 1

Is there consensus? I'd say not. There's a strong community voice for using TDD/BDD but even then it's not universal. The framework is probably less important than applying the practice in the first place.

I believe that minitest is the 1.9 replacement for Test::Unit, which only exists as a wrapper around the new library for backward compatibility. I'd guess that taking the minitest-specific route would be more forward-looking.

RSpec is a somewhat different take: less obviously infused by the "traditional" xUnit-influenced frameworks. It's worth spending some time with to see if you prefer it.

There's also Shoulda. although I don't know how tested it is on 1.9: it's more of a wrapper/extension for Test::Unit, so it'll probably work, though.

Of course the fun doesn't stop there: once you've chosen a test framework you're going to want a mocking engine of some kind. Or a data fixture replacement.

In all cases, I'd say any framework for which you can find evidence of regular updates, an open, populated community and an API you don't hate will do the job. It's unlikely that you're going to be shot down later for having made whatever choice you do.

Answer 2

I have been using Rspec for a year or so. I switched from Test::Unit specifically because I could print in a human readable way what is being tested.

It reads like prose. Which is big plus for me. But as with everything results will vary.

describe Model do
  it "should do what I want" do
    this.should == that
    it.should be_nil
    that.should_not be_true
  end
end 

Also reguardless of test framework you should also think about a coverage tool like RCov. It integrates with rspec and will tell you where your code lacks testing.

Answer 3

RSpec is the consensus of popular Rails books like "The Rails Way". Pros IMHO are tests that read like English, output formats that read like English, and on-the-fly matchers. Cons are slow speed and the library sometimes feels "magical" i.e. too foreign to coders who are more familiar with Ruby and unit tests. RSpec also offers Cucumber and Gherkin, which are ways to write users stories that are useful customer acceptance.

MiniTest is built into Ruby 1.9. Pros IMHO are fast speed, very easy to read/write for coders of other languages, and tests that use much the same code style you'd use in your real code. MiniTest has much more than most people know, for example MiniTest Spec has BDD matchers akin to RSpec, MiniTest Mock is a mocking system, and MiniTest Benchmark is for performance profiling.

In my personal experience, people who build Rails apps tend toward RSpec, and people who build pure Ruby code tend toward MiniTest Unit, Spec, Mock, Benchmark, and so on.

Answer 4

I personally perfer rspec. I found that focusing my tests on behaviour really helped me get around what I was trying to do.

With rspec go and get something like Factory Girl or Machinist to generate your test data, don't use fixtures.

With regard to what Mike said, the rspec project is fairly active with new versions being released normally with each version of rails to help it keep up with the development of the framework.

Answer 5

RSpec is nice but I think recently there has been a lot of talk and movement behind Minitest which comes with Ruby 1.9.x. It has a lot of the niceties of RSpec but is much leaner and and concise.