minitest with simplecov

Question

I'm trying to test minitest files like this:

COVERAGE=true ruby -Itest test/views/info_pages_test.rb
COVERAGE=true ruby -Itest test/views/errors_test.rb

now my info_pages_test has 97% coverage and my errors_test has 75% coverage. Together they should cover 100%.. but each time I run the above commands, I get one result or the other 75% or 97%. Is there a way to combine the results of the two test files into one coverage report?

Help would be greatly appreciated!

Here is the top of my minitest_helper.rb file

## SimpleCOV

require 'simplecov'

if ENV["COVERAGE"]

  SimpleCov.start('rails') do

    add_filter "/test/"

  end
  puts "Started SimpleCOV"
end

I also have a .simplecov file in the application root but using it gives me unpredictable results.. I get 100% coverage once in a while and it is unpredictable..

.simplecov file

SimpleCov.use_merging true
SimpleCov.merge_timeout 3600

Answer 1

The problem you are bumping into is that each of those "test suites" will overwrite each other because the suite name (configurable via SimpleCov.command_name 'xyz'), as opposed to when merging for example cucumber and rspec results.

Preferred solution: Generate the coverage report by running the whole test suite at once, using rake test or some other, similar facility.

If you insist on running individual test files you can trick SimpleCov into merging those results instead of overwriting them by supplying a pseudo-random command name, i.e. SimpleCov.command_name "MiniTest #{Time.now}", or (depending on your setup) using ARGV, i.e. SimpleCov.command_name "Minitest #{File.basename(ARGV[1])}". The latter has the advantage of not duplicating results on re-runs of the same test file since those will be overwritten on merge, but may fail when you run all your tests and do not check for the presence of ARGV correctly, or your test framework tampers with ARGV before you can grab it.

Although you can make this work for individual test runs, in general I'd recommend to base coverage reports off full test suite runs only, as the other approaches leave room for error.