Rails performance tests “rake test:benchmark” and “rake test:profile” give me errors

Question

I'm trying to run a blank default performance test with Ruby 1.9 and Rails 2.3.5 and I just can't get it to work! What am I missing here???

rails testapp
cd testapp
script/generate scaffold User name:string
rake db:migrate
rake test:benchmark

-

/usr/local/bin/ruby19 -I"lib:test" "/usr/local/lib/ruby19/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader.rb" "test/performance/browsing_test.rb" -- --benchmark
Loaded suite /usr/local/lib/ruby19/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
/usr/local/lib/ruby19/gems/1.9.1/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:105:in `rescue in const_missing': uninitialized constant BrowsingTest::STARTED (NameError)
    from /usr/local/lib/ruby19/gems/1.9.1/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:94:in `const_missing'
    from /usr/local/lib/ruby19/gems/1.9.1/gems/activesupport-2.3.5/lib/active_support/testing/performance.rb:38:in `run'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:415:in `block (2 levels) in run_test_suites'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:409:in `each'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:409:in `block in run_test_suites'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:408:in `each'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:408:in `run_test_suites'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:388:in `run'
    from /usr/local/lib/ruby19/1.9.1/minitest/unit.rb:329:in `block in autorun'
rake aborted!
Command failed with status (1): [/usr/local/bin/ruby19 -I"lib:test" "/usr/l...]

Answer 1

See this. For a patch, see here.

Answer 2

OK, the problem is that ruby 1.8 used Test::Unit as its default testing library. In ruby 1.9 that got changed to MiniTest.

Versions of ActiveSupport prior to 3.1.0 assumed Test::Unit. It looks like it has been addressed in 3.1.0 and later. If you're using rails 3.0.9 though, it has a dependency on activesupport 3.0.9, so you can't ask for later versions in your Gemfile.

Found this via Google. Apparently, the Test::Unit library is still available, but as a gem named 'test-unit'.

So in my Gemfile at the top before the rails gem, I put:

gem 'test-unit', '~> 2.3.0'

Then I ran rake test:benchmark and it worked!

Answer 3

I think it's a bug. I am able to run the code with ruby 1.8.6 (2008-08-11 patchlevel 287) [universal-darwin9.0]. Under Ruby 1.9.1, I get an exception like the one you describe.

When it reaches ActiveSupport::Testing::Performance#run (in rails) it causes an exception at the line yield(self.class::STARTED, name) where it tries to find the constant ::START. Ruby Prof has a similar method too, under RubyProf::Test#run.

I don't know who's at fault here.