Valery Komarov

Apr 26 2022

Tags:

#Cpp #GameDev

Unreal baselining: PVS-Studio's enhancements for Unreal Engine projects

• Baselining
• PVS-Studio and Unreal Engine
• Peculiarities of UE projects
• Baselining enhancements that PVS-Studio made for UE projects
• Suppression mode for Unreal Engine projects in plugins for Visual Studio and JetBrains Rider
• Shared use of the new baselining features for UE projects in CI and Visual Studio
• Conclusion

The PVS-Studio static analyzer is constantly evolving. We enhance various mechanisms, integrate the analyzer with game engines, IDEs, CI/CD instruments, and other systems and services. A few years ago, PVS-Studio learned to analyze Unreal Engine projects. However, one of the important mechanisms — suppression of analyzer messages (baselining, disabling warnings for existing code) — didn't work with Unreal Engine projects. This article is about why we didn't implement it immediately and how we finally did it.

Baselining

In PVS-Studio, baselining is a mechanism that allows to 'draw a line' for the analyzer warnings issued during the analysis. The analyzer uses this 'line' to show the user only the warnings issued after it.

The message suppression mechanism is implemented via suppress files. These files contain information about each suppressed analyzer message. These files are taken into account in subsequent analyzer runs. Suppressed warnings don't appear in the final report.

This means, you can only get warnings issued for new or modified code. This mechanism is especially relevant when you integrate PVS-Studio into an existing project. In this case, the first analysis may yield a huge number of warnings. Usually, the number of errors depends on the size of the codebase. Instead of processing these warnings all at once, you can just suppress them. As a result, you'll see only 'fresh' warnings in the next analysis.

You can always redisplay suppressed messages. The PVS-Studio plugins for IDEs also have these features — you can use mass suppression and later select and redisplay specific warnings.

I also recommend reading "How to introduce a static code analyzer in a legacy project and not to discourage the team". It describes the situation above in more detail.

PVS-Studio and Unreal Engine

At the moment, there are several ways to check Unreal Engine projects with PVS-Studio. One of the most convenient options is a plugin for Visual Studio. Right there in the IDE, you can:

• run a UE project's analysis;
• navigate through the analyzer messages;
• process analyzer warnings while writing code.

To get these features, you need to perform a couple of small steps:

• open the NMake tab and in the properties of the UE project, find the build/rebuild command. Add flag '-StaticAnalyzer=PVSStudio' to these commands. Note: if you add this flag, the analysis will run but the project won't be built/rebuilt;
• in the plugin, enable automatic load of the analyzer report for UE projects: PVS-Studio > Options > Specific Analyzer Settings > Save/Load (analyzer report) > AutoloadUnrealEngineLog.

After that you can analyze UE projects in Visual Studio:

• To analyze the whole project, rebuild it. The analysis results will be saved to the report. The path to this report is displayed in the console output window. The PVS-Studio plugin automatically uploads the resulting report to the analyzer messages tab.
• To analyze files changed since the last analyzer run, build your solution (click Build Solution). Note that the file won't be analyzed if the only file changed is the .h plugin file.

The same features are available for the PVS-Studio plugin for Rider.

Then why, with all these capabilities, couldn't PVS-Studio support baselining for UE projects until release 7.16? More than that, in 7.16, the only enhancements we made were in the PVS-Studio_Cmd.exe. command line utility. In 7.17, we implemented baselining for plugins for Visual Studio and JetBrains Rider.

To understand the reasons, let's talk about peculiarities of UE projects.

Peculiarities of UE projects

You may think that to support baselining in UE projects, we only needed to reuse the existing mechanism. It's not that simple. Let's look at the mechanisms we use in the PVS-Studio's analysis (not only for UE projects):

• Before analyzing C++ (not UE) projects, the analyzer collects information about the project. This is necessary for the most exhaustive and highest-quality analysis. The build system (for example, MSBuild) provides such information.
• A specialized build system — Unreal Build Tool (UBT) — builds UE projects. UBT integrates over MSBuild, which is used in Visual Studio and JetBrains Rider.
• Wrapper projects created by MSBuild for Unreal Engine do not contain all the necessary information for complete analysis.
• Because of the point above, we had to integrate the C++ analyzer core (PVS-Studio.exe) directly into UBT. This way we get all the necessary information from the build tool.
• The baselining mechanism is tied to MSBuild projects and the PVS-Studio_Cmd.exe utility, and not to PVS-Studio.exe.
• IDE plugins also relate to the baselining mechanism in PVS-Studio_Cmd.exe.

Thus, significant differences of UE projects from common C++ projects prevented us from reusing baselining in console utility and plugins for IDEs. Besides, integration of the C++ core (PVS-Studio.exe) in UBT imposed additional restrictions on UE projects analysis ran from PVS-Studio_Cmd.exe. These restrictions also affected plugins for IDEs since these plugins interact with this console utility.

However, we could avoid some restrictions. These are:

• the '-StaticAnalyzer=PVSStudio' flag allowed the analysis to be run via PVS-Studio.exe from IDE plugins when the project is built/rebuilt;
• subscription to the build event allows to automatically upload the analysis results in IDE plugins.

Let's see what changes we had to make in the PVS-Studio_Cmd.exe utility and plugins for Visual Studio and JetBrains Rider to support baselining in UE projects.

Baselining enhancements that PVS-Studio made for UE projects

In the 7.16 release, the baselining mechanism was expanded in PVS-Studio_Cmd.exe. First, we added the ability to filter analyzer messages from a report (plog file) with a suppress file. Second, we expanded the mass suppression mechanism. Now you can use it with reports for UE projects. This enhancement allows baselining if there's no need (or possibility) to use the UI. For example, when you integrate PVS-Studio into various CI/CD (Travis Ci, Circle CI, Gitlab, Azure).

The analyzer could suppress messages directly from the PVS-Studio's report (plog) files before the described enhancements. The message suppression mode adds messages from the analyzer's report to the suppress files of a project/solution. PVS-Studio can accomplish this without running the analysis. Below is an example of a command to suppress all messages from the report:

PVS-Studio_Cmd.exe -t path/to/solution/file -a SuppressOnly -o path/to/report 
-u path/to/suppress/file

You can activate this mode via the '-a SuppressOnly' flag. In the optional '-u' flag, it is also possible to specify a path to the suppress file not related to the project. This will (re)write the suppress file along this path, and all messages from the report, that was passed in the '-o' flag, will be added to this suppress file. The same goes for the project's/solution's suppress files.

Besides, we added a mode for filtering messages from the analyzer report — FilterFromSuppress. Its difference from the SuppressOnly mode is that FilterFromSuppress creates a filtered report based on those passed in the '-o' flag. As a result, the filtered report will contain warnings that weren't found in suppress files of a solution, projects, or suppress file from the '-u' flag. This report is saved in the .plog format with the '_filtered' postfix. It's stored near the report from the '-o' flag. Here's an example of a command to filter the report:

PVS-Studio_Cmd.exe -t path/to/solution/file -a FilterFromSuppress ^ 
                   -o path/to/report -u path/to/suppress/file

Flags used for this mode are similar to those in the SuppressOnly mode. To activate the FilterFromSuppress mode, use the '-a FilterFromSuppress' flag.

Both modes provide additional features for working with the analyzer's reports not only for UE projects. However, it's not that simple here either. The baselining mechanism cannot create and use suppress files of the UE projects. Given this feature, to make baselining work, you either have to add a solution-level suppress file or pass it in the '-u' flag when using the SuppressOnly or FilterFromSuppress modes.

If you want to know more about these and other features of baselining for UE projects, you can read documentation (section "Baselining analysis results in Unreal Engine projects").

These enhancements allow you to use baselining mechanism for UE projects on a regular basis. The algorithm for using the suppression mechanism for UE projects now looks like this:

• get a report for a UE project via one of the ways described in the documentation (section "Analyzing projects");
• use PVS-Studio_Cmd.exe in the SuppressOnly mode to suppress all warnings from the resulting report;
• in the next analysis, use the previously received suppress file and the FilterFromSuppress mode in PVS-Studio_Cmd.exe to filter messages from the new report.

Suppression mode for Unreal Engine projects in plugins for Visual Studio and JetBrains Rider

Starting from PVS-Studio 7.17, PVS-Studio plugins for Visual Studio and JetBrains Rider can suppress analyzer messages for UE projects. To begin, add a solution-level suppress file.

Microsoft Visual Studio

In Visual Studio, open the Solution Explorer context menu, right-click the solution and create a new suppress file: Add > New Item... > PVS-Studio Suppression File:

In the window with the analysis results, click 'Suppress All Messages'. Messages will be suppressed if UE projects are built with log auto-loading enabled:

You can use the context menu to suppress only the selected messages:

To view the list of all suppress files used in the solution opened in Visual Studio, go to Extensions > PVS-Studio > Suppress Messages:

This window has several buttons:

• Suppress All - suppresses all messages in the analysis results window;
• Suppress Filtered - suppresses only filtered messages;
• Un-suppress from Selected - deletes the suppress files selected in the window. In this case, all messages from deleted suppress files will appear in the table with the results of the analysis.

JetBrains Rider

The plugin for Rider is younger. It employs a different mechanism for interacting with the PVS-Studio's core. That's why now (release 7.17), you can only suppress all messages received during the analysis of a UE project. To do this, click "Suppress All Messages", just like in Visual Studio.

Shared use of the new baselining features for UE projects in CI and Visual Studio

Taking into account all the improvements described above to the baselining mechanism for UE projects, you can now quickly integrate PVS-Studio into a new project, configure its automatic daily check in CI and at the same time gradually deal with suppressed warnings in Visual Studio.

It looks like this:

• After integrating PVS-Studio into your UE project, you get the first report from the analyzer.
• Next, suppress all warnings via PVS-Studio_Cmd.exe in the SuppressOnly mode or via the PVS-Studio plugins for IDEs. When suppressing all warnings in an IDE, add a solution-level suppress file with all suppressed warnings recorded. As a result, you'll get a suppress file which will further be used to filter warnings from the analyzer reports.
• Commit this suppress file to a version control system or save in any way suitable for you.
• Next, configure a task in CI. This task runs the project analysis and filters warnings from the resulting report. The filter uses the previously obtained suppress file. PVS-Studio_Cmd.exe in the FilterFromSuppress mode filters the messages. After the task is completed, you get a report with warnings issued only for new or changed code.
• After that the report is sent to the team members via the BlameNotifier.exe utility;
• Each team member corrects the code based on the received report.
• Your team can also return to the suppressed messages and gradually fix the legacy code. This requires enabling the display of the suppressed warnings in Visual Studio. After a warning is processed, delete it from the suppress file and commit the change. Thus, this warning won't bother you when you use this suppress file to filter subsequent reports.
• Besides deleting, you can also add new warnings to the suppress file. Access this feature in the interface of the PVS-Studio plugin for Visual Studio or use PVS-Studio_Cmd.exe in SuppressOnly mode. Just specify the path to the previously created suppress file in the '-u' flag. This allows you to set warning processing aside for later (but don't overdo it), and configure the report so as to contain only the necessary warnings.

Conclusion

The described upgrades to the baselining mechanism have enhanced how PVS-Studio works with Unreal Engine projects. Now it will be much easier to use the analyzer with existing projects and run PVS-Studio on a regular basis for UE projects in IDE and CI/CD. See for yourself – request a trial license.

#Cpp #GameDev

0

0

0

0

SHARE

Tags:

#Cpp #GameDev

We can email you a selection of our best articles once a month