How to get started writing a code coverage tool?

Question

Looking for books or other references that discuss actually how to write a code coverage tool in Java; some of the various techniques or tricks - source vs. byte code instrumentation.

This is for a scripting language that generates Java byte code under the hood.

Answer 1

Does your scripting language generate bytecode? Does it generate debug metadata? If so, bytecode instrumentation is probably the way to go. In fact existing tools will probably work (perhaps with minimal modification).

The typical problem with such tools that they are written to work with Java and assume that a class com.foo.Bar.class corresponds to a file com/foo/Bar.java. Unwinding that assumption can be tedious.

EMMA is a ClassLoader that does byte-code re-writing for code-coverage collection in Java. The coding style is a little funky, but I recommend reading source code for some ideas.

If your scripting language is interpreted then you will need a higher-level class loader (at a source level) that hooks into the interpreter.

Answer 2

You can also get the source from a Open Source code coverage tool and learn from it.

Answer 3

Thxm, Mc! http://asm.objectweb.org/ is another one. Excellent documentation on byte code instrumentation, but nothing "directly" aimed at writing a coverage tool - just some hints or ideas.

Answer 4

You might also want to use something like BCEL to analyse which lines of source actually exist in the byte-code. You don't want to report that things like blank lines and comments haven't been covered.

Answer 5

If you're talking about ColdFusion (which I assume you are from the tags) then I'm not sure this is doable but I may be very wrong here...

IIRC, When CF compiles it essentially compiles into a interpreted form of the CFML as a plain old java source file, this is then compiled into the class. Therefore, any instrumentation that you may have will apply to the intermediary version rather than the CFML itself.

Saying that though, Adobe have got the CF debugger now which can step though code, so please prove me wrong - I'd love code coverage in CFML.