Our website uses cookies to enhance your browsing experience.
to the top
close form

Fill out the form in 2 simple steps below:

Your contact information:

Step 1
Congratulations! This is your promo code!

Desired license type:

Step 2
Team license
Enterprise license
** By clicking this button you agree to our Privacy Policy statement
close form
Request our prices
New License
License Renewal
--Select currency--
* By clicking this button you agree to our Privacy Policy statement

close form
Free PVS‑Studio license for Microsoft MVP specialists
* By clicking this button you agree to our Privacy Policy statement

close form
To get the licence for your open-source project, please fill out this form
* By clicking this button you agree to our Privacy Policy statement

close form
I am interested to try it on the platforms:
* By clicking this button you agree to our Privacy Policy statement

close form
check circle
Message submitted.

Your message has been sent. We will email you at

If you haven't received our response, please do the following:
check your Spam/Junk folder and click the "Not Spam" button for our message.
This way, you won't miss messages from our team in the future.

Complementing Unit Testing with Static …

Complementing Unit Testing with Static Analysis, with NUnit as an Example

Aug 17 2016

When discussing static analysis tools for C# projects, programmers will often deny the necessity of static analysis arguing that most errors can be caught through unit testing. So, I decided to find out how well one of the most popular unit-testing frameworks, NUnit, was tested and see if our analyzer could find anything of interest there.



NUnit is a popular unit-testing library for .NET projects ported from Java to C#. Its code is open and can be downloaded from the project website http://www.nunit.org/.

It should be noted that JUnit - the project that NUnit was ported from - was created by such renowned programmers as Erich Gamma, a co-author of the textbook on object-oriented design patterns, and Kent Beck, the creator of the test-driven development and extreme programming methodologies. I recall reading his book Test Driven Development By Example once, where he explains test-driven development by the example of creating a unit-testing framework, like JUnit, following all of his methodologies. What I mean to say is that there is no doubt that JUnit and NUnit were developed in keeping with the best traditions of unit testing, which is also confirmed by Kent Beck's comment at the NUnit site: "... an excellent example of idiomatic design. Most folks who port xUnit just transliterate the Smalltalk or Java version. That's what we did with NUnit at first, too. This new version is NUnit as it would have been done had it been done in C# to begin with."

I looked through NUnit's source files: there are piles of tests; it looks like they have tested everything that could be tested. Taking into account the project's great design and the fact that NUnit has been used by thousands of developers over a number of years, I didn't expect PVS-Studio to find a single bug there. Well, I was mistaken: it did find one bug.

About the bug found

It triggered the V3093 diagnostic, which deals with an issue when programmers use operators & and | instead of && and ||. This issue may cause troubles when it is critical that the right part of an expression should not execute under certain conditions. Let's see what this error looks like in NUnit.

public class SubPathConstraint : PathConstraint
    protected override bool Matches(string actual)
        return actual != null &
            IsSubPath(Canonicalize(expected), Canonicalize(actual));
public abstract class PathConstraint : StringConstraint
    protected string Canonicalize(string path)
        if (Path.DirectorySeparatorChar !=
            path = path.Replace(Path.AltDirectorySeparatorChar,

Even if the Matches method receives the value null as the actual parameter, the right operand of the & operator will be evaluated anyway, which means that the Canonicalize method will be called, too. If you look at this method's definition, you'll see that the value of its path parameter is not tested for null and method Replace is called on it right away - this is where NullReferenceException might be raised. Let's try to reproduce this issue. For that purpose, I wrote a simple unit-test:

public void Test1()
    Assert.That(@"C:\Folder1\Folder2", Is.SubPathOf(null));

Now let's run it and here's what we get:


Here it is: NUnit crashed with NullReferenceException. PVS-Studio managed to find a real bug even in such a well-tested product as NUnit is. Note that it was no harder than writing a unit-test: you just run project analysis from the menu and check the grid with the results.


Unit testing and static analysis are not alternative, but complementary software-development strategies [1]. Download PVS-Studio analyzer and run it on your projects to see if it can find errors that tests didn't.


Comments (0)

Next comments next comments
close comment form