Do we still need to do static analysis? [closed]

Question 1

Short answer: Yes.

Long answer: Compilers are indeed getting better at analysing certain kinds of "mistakes", but the "depth" that they work to is typically a lot less than proper tools for this. In particular tools that work across compile units, such as Coverity, which can understand (for example) that a function may return a pointer as NULL, and if it does, your code will crash because you don't check for NULL before accessing the pointer.

Static analysis tools can also do checking for lock usage, which the compiler typically can't.

As to "how effective" it is really does depend on both which tool you are using, what settings, and how well you otherwise test the code. So, "code coverage" comes into it too. Do you go through every branch of your code in your testing? With every possible value that causes a difference in behaviour? Static analysis tools can detect errors that your testing may not cover.

(Obviously, whether it actually makes sense in your particular business is a completely different discussion - that's for your boss, and his/her bosses to decide)

Question 2

It is not that compilers cannot do the analysis that static analysis tools do. The problem is that static code analysis takes a significant amount of time and is usually not needed for every single compilation. Compilers are generally optimized for a balance of code quality and compilation time. If a compiler happens to stumble on an error in your code it will tell you, but it has no time to actively look for bugs.

Question 3

Also, static analysis is important to generate metrics.

Those metrics can show : cyclomatic complexity, the depth of your class inheritance and many others / dependencies graph / % of comments and much (Understand has a complete list of features for example ).

A good point too is that most static analysis tools allows you to add specific rules which can be useful if your project / company have coding rules.

You can also add a "Continuous integration" server checkout-ing your svn/git/other development branch every day and doing the analysis by night. This way, next day you can fix the code that isn't compliant to your ruleset.

Question 4

Yes, it adds more confidence to the code.

MISRA rules are used widely in real time systems. complying to MISRA (by running misra-supported tool) is required by many software auditors.

Also MISRA (or similar standards) define some cumbersome rules to prevent user from doing fancy stuff which might go wrong on some compilers. For example, pointer arithmetic is prohibited by MISRA, but no compiler will give you a warning on it.

EDIT: Another example: The following code is doing what many people don't expect (x != 1.1 because of floating point accuracy).

int main(int agrc, char** argv){

    float x = 1.1;
    if(x == 1.1)
        printf("Yes!\n");
    else
        printf("OMG!\n");
    return 0;
}

This is not detected by compilers (tried it on gcc 4.6 and clang 3.2). But it is detected by almost all the static analysis tools.

Question 5

Good static analysis tools analyze thing like cyclometric complexity and even things like 'code smells' (see Martin Fowler's Refactoring book). Even line counts in a file or in a function are clues that your code can be improved. I don't think Lint covers these areas, though.

Question 6

Are static analysis tools still needed? Absolutely. The warnings emitted by compilers are getting more sophisticated, but there's still the limitation that a compiler generally works on one source file at a time. Each file is compiled (*.c -> *.o), and the resulting object files are linked together at the end. To do truly effective static analysis, you need to look at the whole code base at once, analyzing individual files as well as the interactions between them. Compilers generally aren't designed like that, so a compiler alone usually misses the sorts of things that static analyzers pick up. You wouldn't want to build this functionality into the normal compiler because it's a fairly heavy performance impact. A static analysis run on my current project typically takes 4x the time a normal compile takes. You don't want this extra overhead on every single build, so it's best to farm it off to a separate, specialized tool than can be run when needed (or in the case of some modern compilers, a separate set of command-line options that are disabled by default).

How effective is static analysis? Very. My team finds a large number of potential problems through static analysis tools, most of which would be nearly impossible to find using other methods. It's particularly good at finding complex problems that humans aren't good at spotting, like those that involve the interaction between multiple local and global variables. Even if you have excellent test coverage, static analyzers will find all sorts of things that are difficult to find through testing alone. This is even more true in the embedded world, where testing tends to be more difficult and less automated. In my experiences, static analyzers have even showed us problems that we didn't even know we needed to be testing for in the first place.

I would definitely recommend the use of static analysis tools for any non-trivial software project. I actually run two separate static analyzers (one is built into the compiler suite and the other is a separate utility); you might be surprised what one will catch and the other will miss. I highly recommend that you come up with a customized set of rules/tests for your analysis runs instead of simply adopting a rule set like MISRA. Every project's needs are different, and many industry-wide specs like MISRA include a lot of stuff that isn't needed for most people. The more unnecessary stuff that you're checking for, the more time it takes to analyze, the more false positives you'll have to wade through, etc etc.