Does the result of static code analysis depend on the compiler being used?
At one of the many discussions of static code analyzers, we mentioned our tool PVS-Studio as usual :-) . One of the participants asked: "Does the result of a static code analyzer depend on the compiler being used at all?" The answer to this question is much more complicated than it seemed at first. That is why you see this post.
Visual Studio 2005 and 2008 development environments are no longer supported. You can view the list of supported development environments in the documentation section "System requirements for PVS-Studio analyzer".
A brief version of the answer is as follows. The static code analyzer is a tool that does not depend on the compiler at all and uses its own techniques to parse the code and search for issues in it. But this is in theory and what do we have in practice?
First, static code analyzer developers very often prefer some development tools they are used to and leave aside (of course unfairly) some others. For example, if a tool is designed for a Windows-platform, it will most likely fail to work with GCC. And even concerning the Windows-platform, a tool intended for Microsoft products will scarcely work efficiently with Embarcadero (Borland followers) products. For if we speak, for example, of C++ language, developers of different compilers always have some mutually incompatible features. It might be explained by the C++ standard size or by the intrigues of marketing wars. One way or another, the fact remains.
Second, the static code analyzer might well use the compiler, at least partly. For example, our PVS-Studio analyzer uses the external preprocessor. And of course it impacts its operation.
Third, even using the same development tool (compiler) but of different versions together with the static analyzer might also have some influence. For example, we have tests for PVS-Studio - about 30 applications (90 projects) in the form of the source codes. Full testing of this set with our tool takes an hour and a half in Visual Studio 2005 and two hours in Visual Studio 2008. The difference is 1.4 times! Where does it come from? You can easily find the answer if you just preprocess all the files being tested and watch the result size. Now it is 1278 files. The size of the preprocessed files in Visual Studio 2005 is 4850 Mbytes and in Visual Studio 2008 - 6850 Mbytes. The difference is these very 1.4 times. I.e. the user code being absolutely the same, the difference between the header files determines the difference between the final codes processed by the analyzer of 1.4 times.
So, it turns out that the compiler you use influences very much what the code analyzer tests.