What are the characteristics for a good report generation software for reporting and tracking software benchmarking results?

Question

This is an offshoot question from this answer to a previous question, where http://speed.pypy.org is highlighted as an example having a good presentation.

(However, it appears to me that the project doesn't separate the components of execution/tracking/report-generation/web-service, which makes it harder for other people to adopt.)

I am interested in both the functional and the UI requirements of such software. I hope to be able to choose an existing one based on the criteria so that I can use it in my project.

Right now, the only thing I can think of is that the Execution UI should be similar to a Unit Testing harness, but the Reporting UI should be totally different from the xUnit family of software. Webpages seem to be a better way to navigate through the results.

And, along with some minor ideas:

• There should be a tracking component to track performance changes at all levels
• However, the presentation layer should highlight only "relevant" performance changes, that is, performance drops in important areas that are serious enough to require developers' attention.

I am also interested in whether any of the advices from Edward Tufte can be applied here.