I want to use PVS-Studio in my project. The manager is against it. How to convince them?
- The analyzer is costly
- It's tough to try or integrate the analyzer
- The analyzer displays a lot of warnings
- The analyzer is wrong and does not understand the logic of the code (false positives)
- The analysis will take a lot of time and will slow down the processes
- You need to upload source code to third-party servers
- Why analyze regularly? After all, you can run it just before the release or once a year
- PVS-Studio is not finding an error
- Developers will not use the analyzer
- I failed to find an answer to my question
You decided to integrate PVS-Studio into your project. But suddenly it turns out that the manager is against it, because... because why, actually? Let's try to figure out what to do with potential objections.
The article is divided into sections on purpose: you don't have to read the whole text, but only the topic that concerns you.
The analyzer is costly
Is this really a matter of price?
First of all, you need to understand, is it really about the price? Maybe some other concerns lurk behind it: the slowdown of the development process, integration difficulties, or something else? If so, then these very objections should be treated.
How can PVS-Studio be useful?
To justify the price, you need to understand how PVS-Studio can help in the development process.
The best option is to run the analyzer on the project code and see the issues it finds. You can download a trial version here.
I also recommend reading the following articles:
- "PVS-Studio ROI": in this article we explained why using PVS-Studio in a project is financially beneficial;
- "SAST in Secure SDLC": here we discussed the reasons to integrate SAST solutions and how it helps to reduce financial and reputational risks;
- "Philosophy of static code analysis: we have 100 developers, the analyzer found few bugs, is analyzer useless?": in this article we explained the reasons for using the analyzer regularly and why a one-time check may not show the tool's full potential.
Tell your manager in which projects PVS-Studio has already detected bugs. Those include:
- Unreal Engine;
PVS-Studio still finds errors in their project code, although these projects are developed by professionals.
The full list of checked projects is available here. There you can also find links to articles describing the detected issues.
It's tough to try or integrate the analyzer
In fact, everything is exactly the opposite. To run the analyzer on a project, you need to:
- get a trial key;
- download a distribution;
- perform the analysis.
This page will help you follow the steps above.
You can use a special filter to display the best warnings. If this filter is unavailable for your environment, I recommend you start viewing warnings from the 'High' level of certainty.
The documentation describes how to use the analyzer with different IDEs, CI/CD, as well as options for setting up the analyzer. For more information on introducing PVS-Studio into development processes, check the article "How to introduce a static code analyzer in a legacy project and not to discourage the team".
Any questions? Contact us. We will help you launch the analysis and deal with any possible difficulties.
The analyzer displays a lot of warnings
On the first launch
Did PVS-Studio display a lot of warnings on first launch? It's all right, particularly for huge legacy code projects.
You need to:
- Realize that the code still works with all these warnings.
- Embrace them as a tech debt.
- Prevent new warnings, correct them timely.
While integrating the analyzer, perform baselining. This is what happens next:
- All warnings found in the project code are hidden. We consider them as a tech debt.
- A team will only see warnings found in new or modified code. So, you can fix problems on time.
- You can return to hidden warnings later.
And again, I recommend reading another article: "How to introduce a static code analyzer in a legacy project and not to discourage the team".
Note. If you are analyzing your code for security flaws, I suggest reviewing the most critical warnings (the "High" level) before suppressing all of them.
On the third-party code
One of the stages of configuring the analyzer is to exclude third-party code that you don't plan to fix from the check, in particular:
- automatically generated files;
- third-party library code.
The exclusion of this kind of code from analysis will not only decrease the number of warnings but also reduce the analysis time.
You can read about the way to exclude files and directories from analysis here.
These warnings are irrelevant for the project's code
The analyzer finds issues of different types, but not all of them could be relevant to your project.
- make sure that you only use diagnostic groups that are relevant the project. For example, if you don't need to check compliance with MISRA standards, check that this diagnostic group is disabled;
- some diagnostics, even within the right group, may be irrelevant for your project. For example, you're checking C# code and you don't care if real numbers are compared directly or with the error amount. In this case, it is worth disabling the corresponding diagnostics (V3024) to reduce the number of warnings.
In the documentation for specific IDEs and platforms, we explained how to disable diagnostic groups.
The analyzer is wrong and does not understand the logic of the code (false positives)
Sometimes static analyzers issue warnings for correct code. Such warnings are called false positives.
PVS-Studio has different mechanisms to deal with them. So, you can suppress false positive warnings either individually or by a template. The documentation describes exactly how to do this.
However, it can be useful to know what false positives are and why they can occur.
An error in analyzer logic
Look at this example:
Thread thread = new Thread(threadStartDeleagate); thread.UnsafeStart();
An analyzer warning might look like this: V3082. The 'Thread' object 'thread' is created but is not started. It is possible that a call to 'Start' method is missing.
PVS-Studio was wrong and issued a false positive because it failed to understand the logic of the code.
If you faced a similar situation, contact us. We will enhance the analyzer to prevent issuing warnings on such code in the future.
The analyzer is right, but the warning is false
There is another type of false positives. Let's inspect another example:
So, let's assume the analyzer displayed 2 warnings:
- V3062. An object 'obj' is used as an argument to its own method. Consider checking the first actual argument of the 'SpecialMethod' method.
- V3062. An object 'anotherObj' is used as an argument to its own method. Consider checking the first actual argument of the 'AnotherMethod' method.
The SpecialMethod and AnotherMethod methods are specific to project. In the first case an object can be used as an argument to its own method, while in the second it can't.
So, in the second case the analyzer issued a correct warning, while in the first case it was wrong. If PVS-Studio is unaware of SpecialMethod and AnotherMethod, these calls will be equal for it. That's why in both cases the analyzer will issue a warning.
If this is an only case, then you can suppress a specific warning. If you see this kind of code regularly, it's better to suppress the warning by the template:
After that, the analyzer will not be triggered on such calls.
The documentation describes the ways to suppress the warnings.
The analysis will take a lot of time and will slow down the processes
The analysis time depends on the machine resources and on the size and complexity of the code base.
If it seems like the analysis is taking too long, the analyzer freezes, or something else happens, please contact us, and we'll handle it.
If you just want to reduce the analysis time, try this:
- disable irrelevant diagnostic groups;
- exclude from the analysis files and directories with code that you will not fix;
- use incremental analysis mode to check only new or modified code;
- configure the analysis of commits / pull requests / merge requests;
- analyze code in parallel on multiple machines using distributed build systems.
Disable irrelevant diagnostics
We have already explained above the reasons for this and the ways to do it in the section "The analyzer displays a lot of warnings: but these warnings are irrelevant for the project's code".
Analysis of code modified since the last build (incremental analysis mode)
Incremental analysis allows you to check only those files that have been modified since the last build. This is another way to analyze only new or modified code.
See the documentation for more information about this mode.
Analysis of commits / pull requests / merge requests
You can configure PVS-Studio to check individual commits / PR / MR, not the entire project. This will reduce the analysis time and allow you to find problems earlier.
For more details on this analysis mode, see the documentation.
Note. I still recommend regular analysis of the whole project together with the partial analysis. For example, during night tests.
Distributed analysis using Incredibuild
When you use PVS-Studio with the distributed build system, you can perform analysis on multiple machines at once. This can greatly reduce the analysis time.
In this article, on the example of Unreal Tournament, we explained the change in the project analysis time with and without Incredibuild.
The documentation explains how to set up distributed analysis.
Note. Currently you can use PVS-Studio and Incredibuild to analyze only C and C++ projects.
You need to upload source code to third-party servers
Not really, you don't have to upload the code to third-party servers. PVS-Studio is the on-premises solution. You are free to choose where to perform the analysis:
- on the machines of developers;
- on the local server;
- in the cloud of your own choice.
If necessary, you can work with the analyzer fully offline.
Why analyze regularly? After all, you can run it just before the release or once a year
The later you find the problem, the harder and more expensive it is to fix it. This is particularly true for security defects. The relative cost of fixing a problem at different stages of the software life cycle, according to the IBM System Science Institute, is as follows:
Defects found after the release are 15 times more expensive than those discovered at the development stage. Moreover, they are 100 times more expensive than defects discovered at the design stage.
Regular static analysis allows you to follow the shift-left principle, find problems earlier and thus vastly reduce development costs. Learn more on this topic here.
PVS-Studio is not finding an error
There may be several reasons for not detecting an error:
- this functionality is not implemented in the analyzer;
- the diagnostics that searches for this problem is off;
- the folder with the analyzed code is excluded from the analysis;
If you have any problems, just contact us and we'll handle it.
If you are considering trying the analyzer on a synthetic test, I suggest you read this article in advance: "Why I Dislike Synthetic Tests". And remember that often the real mistakes are simple enough. For example, typos in C++ (link) and C# (link) projects.
Developers will not use the analyzer
A static analyzer needs to be introduced into the development process in such a way that:
- everyone understands the benefits it provides;
- the analyzer simplifies the development process rather than complicates it.
It is difficult to offer any practical advice on integration of the analyzer because a lot depends on the way the development processes in your team are organized. General recommendations may be as follows:
- perform baselining. So, you will be focusing only on the new warnings, without any distractions on a legacy code;
- preconfigure the analyzer. Exclude unnecessary files from the analysis, disable irrelevant diagnostics, etc. This will reduce the number of redundant warnings;
- install the analyzer on the developers' machines. So, they can find and fix problems much easier and faster;
- regularly analyze the project in CI and correct any issues found timely;
- notify your team of the analysis results. It allows managers to monitor the quality of the project and developers to know about warnings they may have missed. Read more here.
If the tool is beneficial for your project and is properly integrated into the development process, then there is no question about the purpose of its use.
I failed to find an answer to my question
This article will be updated and supplemented. If you can't find the answer to your question, please contact us! :)