To get a trial key
fill out the form below
Team License (standard version)
Enterprise License (extended version)
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Request our prices
New License
License Renewal
--Select currency--
USD
EUR
GBP
RUB
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
To get the licence for your open-source project, please fill out this form
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
To get the licence for your open-source project, please fill out this form
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
I am interested to try it on the platforms:
* By clicking this button you agree to our Privacy Policy statement

** This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Message submitted.

Your message has been sent. We will email you at


If you haven't received our response, please do the following:
check your Spam/Junk folder and click the "Not Spam" button for our message.
This way, you won't miss messages from our team in the future.

>
>
Comparison of static code analyzers: Cp…

Comparison of static code analyzers: CppCat, Cppcheck, PVS-Studio and Visual Studio

We have carried out a thorough comparison of four analyzers for C/C++ code: CppCat, Cppcheck, PVS-Studio and Visual Studio's built-in analyzer. It is a serious, large investigation that we had spent about 170 man-hours on and which, in our opinion, gives a good idea of the general state of things in static analysis nowadays.

Unfortunately, we are no longer developing or supporting the CppCat static code analyzer. Please read here for details.

0241_Short_Comparison_CppCat_Cppcheck_PVS-Studio_Visual_Studio/image1.png

About the comparison

We picked out over ten open-source test projects for the investigation.

Each analyzer was used with the recommended settings of severity levels of diagnostic messages:

  • CppCat: all diagnostics enabled (no severity levels);
  • Cppcheck: Errors and Warnings;
  • PVS-Studio: the 1-st and 2-nd level of general diagnostics;
  • Visual Studio: Microsoft Native Recommended Rules.

We closely studied the analysis reports and picked out those warnings which seemed to indicate genuine bugs or code fragments which at least required close examination. The summarized results are presented in this article.

The comparison methodology is described in detail in the article: "How we compared code analyzers: CppCat, Cppcheck, PVS-Studio and Visual Studio". In that article, you will also find the list of the test projects, the list of the diagnostic messages we picked out, examples of detected bugs and so on. You will also learn from that article why we didn't provide the information about the number of false positives.

Comparison results

The results of the comparison of the analyzers CppCat, Cppcheck, PVS-Studio and Visual Studio's built-in analyzer are presented in three tables:

0241_Short_Comparison_CppCat_Cppcheck_PVS-Studio_Visual_Studio/image2.png

Table 1 - The number of detected genuine bugs (numerical form)

0241_Short_Comparison_CppCat_Cppcheck_PVS-Studio_Visual_Studio/image3.png

Table 2 - The number of detected genuine bugs (percentage form)

0241_Short_Comparison_CppCat_Cppcheck_PVS-Studio_Visual_Studio/image4.png

Table 3 - Analysis time (in minutes)

Analysis time

We don't think analysis time to be a significant comparison metric. But people often wonder about our tools' performance, so we decided to include these figures into the article.

As you can see, Cppcheck is the fastest analyzer. But notice also that the table shows the total analysis time. On some projects, Cppcheck was running slower than the other analyzers. So the actual analysis speed rather depends on a particular project.

CppCat is actually a bit faster than PVS-Studio as it has fewer diagnostic rules to check. But the difference is really insignificant (a few dozens of seconds), so we found it pointless to show it in the table.

Diagnostics

Our team has selected a total of 965 unique code fragments that require investigation and fixing.

As you can see from the table, CppCat and PVS-Studio have shown identical results. The reason is that the PVS-Studio analyzer with recommended settings uses the same set of diagnostics as CppCat.

The total number of defects detected by the analyzers is 742 for CppCat and PVS-Studio each; 193 for Cppcheck; 116 for Visual Studio.

Thus, CppCat and PVS-Studio detect 4 times more errors than Cppcheck and 6 times more errors than Visual Studio's built-in analyzer.

Conclusion

The CppCat and PVS-Studio analyzers have proved to be evident leaders in the number of revealed genuine and potential defects.

References

Popular related articles
Appreciate Static Code Analysis!

Date: Oct 16 2017

Author: Andrey Karpov

I am really astonished by the capabilities of static code analysis even though I am one of the developers of PVS-Studio analyzer myself. The tool surprised me the other day as it turned out to be sma…
Static analysis as part of the development process in Unreal Engine

Date: Jun 27 2017

Author: Andrey Karpov

Unreal Engine continues to develop as new code is added and previously written code is changed. What is the inevitable consequence of ongoing development in a project? The emergence of new bugs in th…
Characteristics of PVS-Studio Analyzer by the Example of EFL Core Libraries, 10-15% of False Positives

Date: Jul 31 2017

Author: Andrey Karpov

After I wrote quite a big article about the analysis of the Tizen OS code, I received a large number of questions concerning the percentage of false positives and the density of errors (how many erro…
PVS-Studio ROI

Date: Jan 30 2019

Author: Andrey Karpov

Occasionally, we're asked a question, what monetary value the company will receive from using PVS-Studio. We decided to draw up a response in the form of an article and provide tables, which will sho…
Technologies used in the PVS-Studio code analyzer for finding bugs and potential vulnerabilities

Date: Nov 21 2018

Author: Andrey Karpov

A brief description of technologies used in the PVS-Studio tool, which let us effectively detect a large number of error patterns and potential vulnerabilities. The article describes the implementati…
Free PVS-Studio for those who develops open source projects

Date: Dec 22 2018

Author: Andrey Karpov

On the New 2019 year's eve, a PVS-Studio team decided to make a nice gift for all contributors of open-source projects hosted on GitHub, GitLab or Bitbucket. They are given free usage of PVS-Studio s…
PVS-Studio for Java

Date: Jan 17 2019

Author: Andrey Karpov

In the seventh version of the PVS-Studio static analyzer, we added support of the Java language. It's time for a brief story of how we've started making support of the Java language, how far we've co…
The Last Line Effect

Date: May 31 2014

Author: Andrey Karpov

I have studied many errors caused by the use of the Copy-Paste method, and can assure you that programmers most often tend to make mistakes in the last fragment of a homogeneous code block. I have ne…
The Ultimate Question of Programming, Refactoring, and Everything

Date: Apr 14 2016

Author: Andrey Karpov

Yes, you've guessed correctly - the answer is "42". In this article you will find 42 recommendations about coding in C++ that can help a programmer avoid a lot of errors, save time and effort. The au…
The Evil within the Comparison Functions

Date: May 19 2017

Author: Andrey Karpov

Perhaps, readers remember my article titled "Last line effect". It describes a pattern I've once noticed: in most cases programmers make an error in the last line of similar text blocks. Now I want t…

Comments (0)

Next comments

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
This website uses cookies and other technology to provide you a more personalized experience. By continuing the view of our web-pages you accept the terms of using these files. If you don't want your personal data to be processed, please, leave this site.
Learn More →
Accept