What are my alternatives for test code coverage on Android?

Question

I know Android's Ant tooling comes with built-in Emma code coverage, but Emma's reports look more and more dated and cannot be easily integrated into other coverage reports or even Sonar, basically because Emma lacks a detailed XML reporting format. What I want to achieve is

• code coverage for unit tests (Robolectric-based, already done with Cobertura)
• code coverage for integration tests (Robotium-/emulator-based, currently done separately with Emma)
• ideally merging code coverage results (this is IMHO only possible with Cobertura and requires the raw *.ser files from every test execution)
• publishing to Sonar (currently only done with the unit test coverage, Sonar understands only one coverage format to my knowledge)

Do I have any alternatives?

edit: This post on sonar-devel probably sums up the current state of Android code coverage support as well as Sonar support pretty much. sigh

Answer 1

I have pushed a github repo that contains the same configuration as you mentionned in your question :

• standard android testing + emma code coverage
• robolectric + cobertura code coverage (btw thx, I thought cobertura was not compatible with robolectric)

and their inclusion in sonar. Only one at a time, but still the configuration is an interesting base to work on.

https://github.com/stephanenicolas/Quality-Tools-for-Android

A colleague and I have in mind to work on this issue with a very similar mindset as you do. We want to get better integration of different testing technologies to be available for Android projects and reporting to work in sonar as well.

If you want to join our effort, we would be happy to give you the rights to the repo and start working with you.

Stéphane

-- Update

Since march 2013, we also propose a unified configuration to use Jacoco offline instrumentation for both robolectric and standard junit tests and get results displayed simultanously in Sonar.

Answer 2

You can use the robolectric framework. It will allow you to run JUnit Tests on the Java VM instead of using Android Junit Tests that run on the Dalvik VM. This will allow you to use any standard coverage Tool out there (EclEmma, CodeCover, Cobertura, etc.). It's easy to configure and runs stable (What you can't say from the Clover tool).

Answer 3

Disclaimer: I'm an Atlassian

Yes. There is an alternative - you can use the Atlassian Clover tool. There is an alpha version of Clover-for-Android available from https://confluence.atlassian.com/display/CLOVER/Clover-for-Android

You wrote that you want to achieve the following:

• code coverage for unit tests - yes, Clover supports "Run as > Android JUnit Test" as well as "Run as > JUnit Test" - i.e. without starting an emulator (but the latter would require slightly different setup as described in the Clover-for-Android manual)
• code coverage for integration tests - as above, "Run as > Android JUnit Test"
• ideally merging code coverage results - yes, you can merge code coverage results from multiple test runs; what's more you can even run your application in emulator or device and perform manual tests (sic!)
• publishing to Sonar - there is a Clover Sonar Plugin available, however I didn't test it with Clover-for-Android; probably you would have to run your tests in Eclipse, next export XML or HTML report (button in Clover's Coverage Explorer view) and push these files to Sonar build