Grandma's recipe for mastering regular static analysis

• Incremental analysis in the development process
• Incremental analysis in CI/CD
• Pull request analysis
• Is it necessary to analyze the whole project?
• Summary

Static analysis is important to perform regularly, but what if it takes an eternity for a whole project? The article addresses this question and provides step-by-step recipes for handling specific cases.

Static analysis helps detect errors and vulnerabilities in source code, but you gain the most benefit only from its regular use. While using the analyzer regularly, you may encounter another problem: the large project may take too long to be analyzed.

There are several ways to address this issue; for example, you can use the incremental analysis mode. It enables us to analyze only the code that has been added or modified since the last analyzer run.

Incremental analysis in the development process

Let's imagine some developers working on a large project. After making some changes to the code, they want to ensure no new errors have been introduced. In that case, why would developers need to fully analyze the codebase if they're only working on a very specific feature in a very specific part of the project? It would be more efficient to focus on the modified code fragments, right?

For such scenarios, PVS-Studio has a special setting in IDE plugins. For example, in the Visual Studio IDE, there is the Extensions > PVS-Studio > Analysis after Build item (Modified files only) tab.

When enabled, only the modified files are analyzed after each successful project build in Visual Studio. The PVS-Studio icon in the system tray indicates that the analysis is in progress.

The PVS-Studio plugin for IntelliJ IDEA has a similar mode. To enable automatic incremental analysis after each build, check the corresponding box in the plugin settings:

You can also automate incremental analysis in the PVS-Studio plugin for Qt Creator.

Each developer using PVS-Studio on their machine can promptly react to the analyzer warning and fix errors before moving on to next development stages.

Incremental analysis in CI/CD

Static analyzers in a CI/CD pipeline enable developers to automate static analysis and ensure it become a regular part of the development process. In CI/CD, it's also not necessary to analyze the entire codebase because you know that all the previous code fragments are fully operational—all that's left is to check the last commit.

Moreover, if you integrate static analysis into the pipeline, you'll also have unit tests, which, even with perfect unit distribution, integration and functional tests, will consume time. Therefore, the use of incremental analysis can help not only in getting quick results but also in accelerating the pipelines.

The approach to enabling incremental analysis depends on the utility you're using. Let's explore how it works in each case.

pvs-studio-analyzer, CompilerCommandsAnalyzer, CLMonitor (Windows, Linux, macOS)

To enable the incremental analysis mode in CMake projects, add a special flag to the command:

pvs-studio-analyzer analyze ... \
  --incremental ...

For other compilers, use a compiler monitoring system.

On Linux, use the trace mode in the same pvs-studio-analyzer flag. In this mode, the analyzer monitors and logs child processes of the build system to detect the compilation process among them. To check only the latest changes, use the incremental build and then run the analysis:

pvs-studio-analyzer trace \
  -- %command_to_build_project%
pvs-studio-analyzer analyze ...

On Windows, use the CLMonitor.exe utility to monitor compiler runs. In this case, similar to pvs-studio-analyzer in the trace mode, you can use the incremental build mode to check only the new code during the monitoring compilation process.

PVS-Studio_Cmd and pvs-studio-dotnet (Windows, Linux, macOS)

To analyze MSBuild projects, use the command line utilities, PVS-Studio_Cmd.exe (Windows) and pvs-studio-dotnet (Linux, macOS). These utilities also include the -incremental flag, but it has the following additional settings:

• Scan: Scans all dependencies to determine which files need to be analyzed. This step must be performed before the project build. The scan results will be saved to temporary .pvs-studio directories and the analysis won't be performed in this mode.
• AppendScan: Works similarly to Scan, but considers not only changes made since the last build but also all previous changes. Meanwhile, Scan considers only the changes made since the last build;
• Analyze: Runs the analysis. Prior to this, you should run the analysis in the Scan mode. Only the files that have changed since the last analysis run will be analyzed.
• ScanAndAnalyze: Combines the settings above, i.e. it first scans which files in the project have changed, and then analyzes them.

pvs-studio-java

For Java projects, you can enable incremental analysis using a special checkbox in the build scripts.

If you use the Maven plugin:

<plugin>
  <groupId>com.pvsstudio</groupId>
  <artifactId>pvsstudio-maven-plugin</artifactId>
  ....
  <configuration>
    <analyzer>
       ....
       <incremental>true</incremental>
       ....
    </analyzer>
  </configuration>
</plugin>

If you use the Gradle plugin:

apply plugin: com.pvsstudio.PvsStudioGradlePlugin
pvsstudio {
  ....
  incremental = true
  ....
}

Pull request analysis

In PVS-Studio version 7.34, the PVS-Studio_Cmd (Windows) and pvs-studio-dotnet (Linux, macOS) command line utilities now include the mode for checking modified files, which serves as an analog of incremental analysis.

The analyzer gets information about file modifications and saves it to a compilation dependency cache. The dependency cache is a list of project source code files and their dependencies.

In the modified file analysis mode, the hash of each source file in the cache is evaluated. On the next analysis run, the hash in the dependency cache is compared to the actual hash value of the file. If the hashes differ, the analyzer will check this file.

You can enable the modified file analysis mode via the ‑‑analyseModifiedFiles (-F) flag. Here's an example of a command to use this analysis mode:

PVS-Studio_Cmd.exe -t MyProject.sln -F ^
                   -D path\to\depCacheDir ^
                   -R depCacheRoot ^
                   -o analysis_report.json

If the compilation dependency cache file doesn't exist, it'll be created, initiating the analysis of all source files in the project. To avoid that, run the first analysis in the dependency cache file generation mode to evaluate hashes for the source code files (the -H flag) without starting the analysis (the-W flag). To do this, use the following command:

PVS-Studio_Cmd.exe -t MyProject.sln -W -H ^
                   -D path\to\depCacheDir ^
                   -R depCacheRoot

This generates dependency cache files but doesn't start the analysis. On the next run of modified file analysis, only newly added or modified source code files will be analyzed.

Let's run an experiment to see how much time this approach can save—I used the ScreenToGif project, which we recently checked for bugs.

I got the following results: the full project analysis took 87 seconds, generating the dependency cache files took 14 seconds, and analyzing only the modified files took 17 seconds.

Thus, the regular analysis time reduced from 87 seconds to 31 seconds—it's almost a threefold difference! Moreover, the project used for the experiment is relatively small, so the difference can be truly dramatic in larger projects.

This mode is useful for analyzing commits and pull/merge requests when you've already ensured that everything is OK in the main repository branch. It helps you check new code for a cruel bug that you might have to fix later.

Use the following algorithm to check a commit or merge request:

• Run the compilation cache generation for the source branch.
• Get and apply changes from the commit or merge request.
• Run modified file analysis.

Another option for checking pull requests or commits with PVS-Studio is to use the file list checking mode in the command line utilities (PVS-Studio_Cmd.exe, pvs-studio-analyzer, or CompilerCommandsAnalyzer.exe).

To use the file list checking mode, specify a special flag:

• ‑‑sourceFiles or -f for PVS-Studio_Cmd.exe;
• ‑‑source-files or -S for pvs-studio-analyzer.

Then specify the path to a file containing a list of paths to the source code files you want to analyze.

On the first analysis run with these flags, the analyzer builds the dependency cache by adding the specified files and their dependency files. Dependency information is stored in the separate dependency cache file for each project. On each subsequent run of the file list checking mode, the information in the dependency cache will be added or updated for:

• files of source code in the file list;
• files with missing entries in the dependency cache;
• files previously dependent on the files in the list.

After updating the dependency cache, the analyzer will scan all files in the file list as well as any files dependent on them.

To maintain the dependency cache up-to-date, run the analysis with this flag each time you make changes to the source files. You can also do this all at once for all changed files from multiple commits. Skipping the analysis of source file changes or running separate analyses of modified files in an order different from their modification order may cause some files to be skipped. This happens when changes in the project's dependency structure affect the analysis process.

If it's impossible to guarantee transfer of all modifiable files in a project, PVS-Studio has flags for regenerating dependency caches.

To regenerate the cache with running the analysis, use:

• ‑‑regenerateDependencyCache or -G for PVS-Studio_Cmd.exe;
• ‑‑regenerate-dependency-info for pvs-studio-analyzer.

To regenerate the cache without running the analysis, use:

• ‑‑regenerateDependencyCacheWithoutAnalysis or -W for PVS-Studio_Cmd.exe;
• ‑‑regenerate-dependency-info-without-analysis for pvs-studio-analyzer.

These flags allow you to keep the cache up-to-date by regenerating it from scratch when necessary.

Is it necessary to analyze the whole project?

The short answer: yes, it's still necessary. Now, let's dive into details.

The cases discussed earlier emphasize the need for the fastest possible analysis. Using abstract examples, we've confirmed that developers should leverage incremental analysis or similar modes for efficiency. However, the analysis of the whole project is also necessary. Once you've analyzed the modified parts of the project, it'd be better to ensure that the overall project quality has not become worse.

For example, a method recently changed by the developer may have been used in another part of the project, and now that part may not function as intended.

A practical approach is to schedule a full project analysis during nightly builds. If serious warnings are detected, you can share the analysis results with the team members responsible for the code. PVS-Studio has a special tool for this purpose, Blame Notifier.

Summary

In this article, we explored scenarios where saving time during regular static analysis is crucial. We reviewed several methods, including incremental analysis, modified file analysis, and file list checking, and demonstrated how you can apply them using the PVS-Studio static analyzer.

To learn more about how to use our analyzer, refer to the documentation.