Pour obtenir une clé
d'essai remplissez le formulaire ci-dessous
Demandez des tariffs
Nouvelle licence
Renouvellement de licence
--Sélectionnez la devise--
* En cliquant sur ce bouton, vous acceptez notre politique de confidentialité

Free PVS-Studio license for Microsoft MVP specialists
To get the licence for your open-source project, please fill out this form
** En cliquant sur ce bouton, vous acceptez notre politique de confidentialité.

I am interested to try it on the platforms:
** En cliquant sur ce bouton, vous acceptez notre politique de confidentialité.

Message submitted.

Your message has been sent. We will email you at

If you haven't received our response, please do the following:
check your Spam/Junk folder and click the "Not Spam" button for our message.
This way, you won't miss messages from our team in the future.

Complementing Unit Testing with Static …

Complementing Unit Testing with Static Analysis, with NUnit as an Example

17 Aoû 2016

When discussing static analysis tools for C# projects, programmers will often deny the necessity of static analysis arguing that most errors can be caught through unit testing. So, I decided to find out how well one of the most popular unit-testing frameworks, NUnit, was tested and see if our analyzer could find anything of interest there.



NUnit is a popular unit-testing library for .NET projects ported from Java to C#. Its code is open and can be downloaded from the project website http://www.nunit.org/.

It should be noted that JUnit - the project that NUnit was ported from - was created by such renowned programmers as Erich Gamma, a co-author of the textbook on object-oriented design patterns, and Kent Beck, the creator of the test-driven development and extreme programming methodologies. I recall reading his book Test Driven Development By Example once, where he explains test-driven development by the example of creating a unit-testing framework, like JUnit, following all of his methodologies. What I mean to say is that there is no doubt that JUnit and NUnit were developed in keeping with the best traditions of unit testing, which is also confirmed by Kent Beck's comment at the NUnit site: "... an excellent example of idiomatic design. Most folks who port xUnit just transliterate the Smalltalk or Java version. That's what we did with NUnit at first, too. This new version is NUnit as it would have been done had it been done in C# to begin with."

I looked through NUnit's source files: there are piles of tests; it looks like they have tested everything that could be tested. Taking into account the project's great design and the fact that NUnit has been used by thousands of developers over a number of years, I didn't expect PVS-Studio to find a single bug there. Well, I was mistaken: it did find one bug.

About the bug found

It triggered the V3093 diagnostic, which deals with an issue when programmers use operators & and | instead of && and ||. This issue may cause troubles when it is critical that the right part of an expression should not execute under certain conditions. Let's see what this error looks like in NUnit.

public class SubPathConstraint : PathConstraint
    protected override bool Matches(string actual)
        return actual != null &
            IsSubPath(Canonicalize(expected), Canonicalize(actual));
public abstract class PathConstraint : StringConstraint
    protected string Canonicalize(string path)
        if (Path.DirectorySeparatorChar !=
            path = path.Replace(Path.AltDirectorySeparatorChar,

Even if the Matches method receives the value null as the actual parameter, the right operand of the & operator will be evaluated anyway, which means that the Canonicalize method will be called, too. If you look at this method's definition, you'll see that the value of its path parameter is not tested for null and method Replace is called on it right away - this is where NullReferenceException might be raised. Let's try to reproduce this issue. For that purpose, I wrote a simple unit-test:

public void Test1()
    Assert.That(@"C:\Folder1\Folder2", Is.SubPathOf(null));

Now let's run it and here's what we get:


Here it is: NUnit crashed with NullReferenceException. PVS-Studio managed to find a real bug even in such a well-tested product as NUnit is. Note that it was no harder than writing a unit-test: you just run project analysis from the menu and check the grid with the results.


Unit testing and static analysis are not alternative, but complementary software-development strategies [1]. Download PVS-Studio analyzer and run it on your projects to see if it can find errors that tests didn't.


Popular related articles
Sorting in C#: OrderBy.OrderBy or OrderBy.ThenBy? What's more effective and why?

Date: 20 Sep 2022

Author: Sergey Vasiliev

Suppose we need to sort the collection by multiple keys. In C#, we can do this with the help of OrderBy().OrderBy() or OrderBy().ThenBy(). But what is the difference between these calls? To answer th…
ML.NET: can Microsoft's machine learning be trusted?

Date: 08 Sep 2022

Author: Andrey Moskalev

In 2018, Microsoft created ML.NET, a machine learning framework for .NET developers. Since then, the machine learning library has undergone significant changes and acquired new features to identify p…
The risks of using vulnerable dependencies in your project, and how SCA helps manage them

Date: 06 Sep 2022

Author: Nikita Lipilin

Most applications today use third-party libraries. If such a library contains a vulnerability, an app that uses this library may also be vulnerable. But how can you identify such problematic dependen…
Build to order? Checking MSBuild for the second time

Date: 01 Sep 2022

Author: Nikita Panevin

MSBuild is a popular open-source build platform created by Microsoft. Developers all over the world use MSBuild. In 2016, we checked it for the first time and found several suspicious places. Can we …
The Orchard Core threequel. Rechecking the project with PVS-Studio

Date: 25 Aoû 2022

Author: Aleksey Avdeev

In this article, we check the Orchard Core project with the help of the PVS-Studio static analyzer. We are going to find out if the platform code is as good as the sites created on its basis. May the…

Comments (0)

Next comments
Unicorn with delicious cookie
Nous utilisons des cookies pour améliorer votre expérience de navigation. En savoir plus