The PVS-Studio static analyzer is constantly evolving. We enhance various mechanisms, integrate the analyzer with game engines, IDEs, CI/CD instruments, and other systems and services. A few years ago, PVS-Studio learned to analyze Unreal Engine projects. However, one of the important mechanisms — suppression of analyzer messages (baselining, disabling warnings for existing code) — didn't work with Unreal Engine projects. This article is about why we didn't implement it immediately and how we finally did it.
In PVS-Studio, baselining is a mechanism that allows to 'draw a line' for the analyzer warnings issued during the analysis. The analyzer uses this 'line' to show the user only the warnings issued after it.
The message suppression mechanism is implemented via suppress files. These files contain information about each suppressed analyzer message. These files are taken into account in subsequent analyzer runs. Suppressed warnings don't appear in the final report.
This means, you can only get warnings issued for new or modified code. This mechanism is especially relevant when you integrate PVS-Studio into an existing project. In this case, the first analysis may yield a huge number of warnings. Usually, the number of errors depends on the size of the codebase. Instead of processing these warnings all at once, you can just suppress them. As a result, you'll see only 'fresh' warnings in the next analysis.
You can always redisplay suppressed messages. The PVS-Studio plugins for IDEs also have these features — you can use mass suppression and later select and redisplay specific warnings.
I also recommend reading "How to introduce a static code analyzer in a legacy project and not to discourage the team". It describes the situation above in more detail.
At the moment, there are several ways to check Unreal Engine projects with PVS-Studio. One of the most convenient options is a plugin for Visual Studio. Right there in the IDE, you can:
To get these features, you need to perform a couple of small steps:
After that you can analyze UE projects in Visual Studio:
The same features are available for the PVS-Studio plugin for Rider.
Then why, with all these capabilities, couldn't PVS-Studio support baselining for UE projects until release 7.16? More than that, in 7.16, the only enhancements we made were in the PVS-Studio_Cmd.exe. command line utility. In 7.17, we implemented baselining for plugins for Visual Studio and JetBrains Rider.
To understand the reasons, let's talk about peculiarities of UE projects.
You may think that to support baselining in UE projects, we only needed to reuse the existing mechanism. It's not that simple. Let's look at the mechanisms we use in the PVS-Studio's analysis (not only for UE projects):
Thus, significant differences of UE projects from common C++ projects prevented us from reusing baselining in console utility and plugins for IDEs. Besides, integration of the C++ core (PVS-Studio.exe) in UBT imposed additional restrictions on UE projects analysis ran from PVS-Studio_Cmd.exe. These restrictions also affected plugins for IDEs since these plugins interact with this console utility.
However, we could avoid some restrictions. These are:
Let's see what changes we had to make in the PVS-Studio_Cmd.exe utility and plugins for Visual Studio and JetBrains Rider to support baselining in UE projects.
In the 7.16 release, the baselining mechanism was expanded in PVS-Studio_Cmd.exe. First, we added the ability to filter analyzer messages from a report (plog file) with a suppress file. Second, we expanded the mass suppression mechanism. Now you can use it with reports for UE projects. This enhancement allows baselining if there's no need (or possibility) to use the UI. For example, when you integrate PVS-Studio into various CI/CD (Travis Ci, Circle CI, Gitlab, Azure).
The analyzer could suppress messages directly from the PVS-Studio's report (plog) files before the described enhancements. The message suppression mode adds messages from the analyzer's report to the suppress files of a project/solution. PVS-Studio can accomplish this without running the analysis. Below is an example of a command to suppress all messages from the report:
PVS-Studio_Cmd.exe -t path/to/solution/file -a SuppressOnly -o path/to/report
-u path/to/suppress/fileYou can activate this mode via the '-a SuppressOnly' flag. In the optional '-u' flag, it is also possible to specify a path to the suppress file not related to the project. This will (re)write the suppress file along this path, and all messages from the report, that was passed in the '-o' flag, will be added to this suppress file. The same goes for the project's/solution's suppress files.
Besides, we added a mode for filtering messages from the analyzer report — FilterFromSuppress. Its difference from the SuppressOnly mode is that FilterFromSuppress creates a filtered report based on those passed in the '-o' flag. As a result, the filtered report will contain warnings that weren't found in suppress files of a solution, projects, or suppress file from the '-u' flag. This report is saved in the .plog format with the '_filtered' postfix. It's stored near the report from the '-o' flag. Here's an example of a command to filter the report:
PVS-Studio_Cmd.exe -t path/to/solution/file -a FilterFromSuppress ^
-o path/to/report -u path/to/suppress/fileFlags used for this mode are similar to those in the SuppressOnly mode. To activate the FilterFromSuppress mode, use the '-a FilterFromSuppress' flag.
Both modes provide additional features for working with the analyzer's reports not only for UE projects. However, it's not that simple here either. The baselining mechanism cannot create and use suppress files of the UE projects. Given this feature, to make baselining work, you either have to add a solution-level suppress file or pass it in the '-u' flag when using the SuppressOnly or FilterFromSuppress modes.
If you want to know more about these and other features of baselining for UE projects, you can read documentation (section "Baselining analysis results in Unreal Engine projects").
These enhancements allow you to use baselining mechanism for UE projects on a regular basis. The algorithm for using the suppression mechanism for UE projects now looks like this:
Starting from PVS-Studio 7.17, PVS-Studio plugins for Visual Studio and JetBrains Rider can suppress analyzer messages for UE projects. To begin, add a solution-level suppress file.
Microsoft Visual Studio
In Visual Studio, open the Solution Explorer context menu, right-click the solution and create a new suppress file: Add > New Item... > PVS-Studio Suppression File:
In the window with the analysis results, click 'Suppress All Messages'. Messages will be suppressed if UE projects are built with log auto-loading enabled:
You can use the context menu to suppress only the selected messages:
To view the list of all suppress files used in the solution opened in Visual Studio, go to Extensions > PVS-Studio > Suppress Messages:
This window has several buttons:
JetBrains Rider
The plugin for Rider is younger. It employs a different mechanism for interacting with the PVS-Studio's core. That's why now (release 7.17), you can only suppress all messages received during the analysis of a UE project. To do this, click "Suppress All Messages", just like in Visual Studio.
Taking into account all the improvements described above to the baselining mechanism for UE projects, you can now quickly integrate PVS-Studio into a new project, configure its automatic daily check in CI and at the same time gradually deal with suppressed warnings in Visual Studio.
It looks like this:
The described upgrades to the baselining mechanism have enhanced how PVS-Studio works with Unreal Engine projects. Now it will be much easier to use the analyzer with existing projects and run PVS-Studio on a regular basis for UE projects in IDE and CI/CD. See for yourself – request a trial license.