Does C1 code coverage analysis exist for Ruby? [closed]

Question

I'm currently using Rcov to get C0 code coverage analysis for a rails project that I'm working on.

However, those results are practically meaningless- I have 100% coverage according to rcov (as it only covers C0 analysis) and I've barely written half the test cases for the functionality that exists thus far.

I'm used to the useful results from the code coverage in Visual Studio 2008 Team, which has C1 coverage. Are there any tools that provide similar coverage for ruby?

Answer 1

At the moment, there are no C1 coverage tools for Ruby. In fact, there aren't any coverage tools other than RCov.

Until recently, it was only possible to write tools like this by patching or extending the MRI interpreter in C. Since approximately two years ago, it is also possible to extend JRuby in Java, and there is actually since last month a port of RCov for JRuby. However, this requires both a knowledge of Ruby and C, and a pretty deep knowledge at that, because fiddling around with the internals of MRI is not for the faint at heart.

But only with Rubinius will it be possible to write dynamic analysis tools such as code coverage tools in Ruby itself, making tool writing accessible to a much larger portion of the Ruby community. My hope is that this, coupled with the substantial financial backing of tool vendors (many major IDE vendors are either working on or have already introduced Ruby IDEs, including CodeGear (ex-Borland), IntelliJ, NetBeans, Eclipse, SapphireSteel (Ruby in Steel for Visual Studio) and even Microsoft) will lead to rapid innovation in the Ruby tooling space in 2009 and we will see things like C1, C2 coverage, NPath complexity, much more fine-grained profiling and so on.

Until then, the only idea I have is to use Java tools. The JRuby guys try to emit the proper magic metadata to make their generated bytecode at least penetrable by the Java tools. So, maybe it is possible to use Java coverage tools with JRuby. However, I have no idea whether that actually works, nor if it is supposed to work.

Answer 2

DeepCover is the only complete C1 Ruby Coverage tool that exists today.

Disclosure: I'm co-author of DeepCover.

Answer 3

I don't know about code coverage tools, but ruby has a mutation tester called heckle.

Mutation testing involves changing the code being tested and seeing if those mutations cause the unit tests to fail.

Unfortunately, the latest official gem (version 1.4.1) has some bugs in it, so you may wish to build a gem using code from the development branch. It can only mutate instance methods. I'm not sure how reliable the development branch version is either.

Mutation testing should be avoided if the code being tested may potentially destroy the system (eg deleting files). You may want to read a FAQ about heckle and a FAQ question mentioning rcov versus heckle.

Answer 4

For the record, NetBeans 7.0 has code coverage analysis for Ruby built-in. I can't find any docs explaining if it's C0 or C1 or not, but it's something to keep an eye on.

Answer 5

Old post, but I ran into it looking for Ruby coverage tools. FYI I ran into a C1 coverage tool for Ruby called single_cov. Don't know how good it it, but I will give it a test-run one of these days.