Question

Supposing you work in a very large global company with lot of code at all levels (e.g. from embedded to mobile platform application code), and projects can last anything between 6 months to 5 years...

What would be considered the best practice in terms of keeping static analysis information?, assuming traceability between defects and static warnings is important for the organization (e.g. aerospace, automotive, government, etc.)

No correct solution

OTHER TIPS

I don't believe that you need to keep much static analysis information, if any at all. However, to support this, you need good configuration management practices.

First, you should be regularly tagging the code in your source code repository. Consider tagging (git, Subversion, ClearCase - most version control systems should support similar functionality) any builds that you do. By tagging crucial builds with a unique identifier (like a build number or timestamp), you can relate things like a particular snapshot of source code to entries in a defect tracking database or static analysis results.

Now that you have tags, you can map static analysis results to specific source code repositories. It's commonly accepted that you shouldn't keep generated code in version control. I would argue that you shouldn't keep anything that can be directly generated from the source code, which includes static analysis. However, for traceability purposes, you will need to be able to associate defects derived from static analysis to the build that caused the finding.

Realistically, you may want to keep some static analysis results. For example, the results of analysis prior to an external release may be included as part of a formal test report. Even if it isn't part of a test report, you may want to keep this in a project archive. You may also consider including a static analysis summary (such as number of findings, number of findings per source line of code, number of findings by module, number of findings by criticality, or number of findings by type).

You may also want to consider how you manage your static analysis tool, especially if you are using it as part of some kind of formal testing or report. You need to be able to consider the version of the tool as well as the configuration file or parameters used when executing the tool over a given version of the code base. If you don't manage your tool version and configuration, it becomes harder to be able to regenerate the exact same report for a given code base.

I find that keeping static analysis results is a very good practice. I wouldn't necessarily be so interested as to "which build introduced this warning", because that should be relatively easy to see from your version control system. In addition, if you are interested in tracing the history of a certain static analysis warning, you probably have already done something else wrong by not fixing that warning when it appeared in the first place.

The main reason why I like the histories of static analysis results is that it allows me to see trends. For example, I can see the amount of lines of code, number of issues, and the code coverage metrics. Issues, for example, should be in a linear relationship with the lines of code. If the number of issues grows faster than the lines of code, someone is probably doing something wrong. Similarly, it helps to establish a baseline of desired test coverage, and you can see if it starts to dip, if someone forgets to create tests.

As to your actual question on how long to keep the static analysis information. I'd say you should keep it right from the beginning of the project, but you don't need to keep all of it. SonarQube, for example, which would be my weapon of choice, automatically prunes some its old snapshots. (A snapshot is a set of static analysis results on a particular build.) The default rules are described at http://docs.sonarqube.org/display/SONAR/History+and+Events and the most relevant part is this:

  • only one snapshot per day is kept after 1 day. Snapshots marked by an event are not deleted.
  • only one snapshot per week is kept after 1 month. Snapshots marked by an event are not deleted.
  • only one snapshot per month is kept after 1 year. Snapshots marked by an event are not deleted.
  • all snapshots older than 5 years are deleted, including snapshots marked by an event.

So for the current day you see all of the analyses, but tomorrow only the last analysis of the previous day is available, the others have been deleted. At the end of April all but the last analysis of March are deleted. Etc. So go for a different resolution for different time periods. This allows you to efficiently save several years of data.

Licensed under: CC-BY-SA with attribution
scroll top