Five days for fixing a two-character error, or a myth of almighty technologies aiding software development

Aug 30 2010

Author: Evgenii Ryzhkov

In this blog, you may often read posts about how this or that software tool or software development technology helps make fewer errors, find them faster and correct them easier. Surely this all is true but no one should ignore the main tool - brain, for nothing can replace it. Today I will speak about a program error that took us two characters and five days of work to fix it.

Once, after we released a new version of the PVS-Studio static code analyzer, a user addressed us with a complaint that our tool would not work on his complex project. By the way, it is a good means for developers to estimate how much a program is useful - if you receive several letters with messages that something is broken and does not work after you release a new version, it means that your program is needed. Let's return to the program. So, if it does not work, it means that it cannot find files for analysis. First we asked the user about the type of his project and plug-ins for Visual Studio he had installed on his computer. It turned out that the user had Intel C++ Compiler but it was not the reason. He also had CUDA installed but this was not the reason either. Yes, by the way, the check of the simple demo project worked well but the check of a new empty project blank did not.

We asked the user to send us this project blank. It was checked correctly on our computers. The matter was additionally complicated by the fact that integration tests, unit-tests, user interface tests and some other tests ran successfully in that version.

By the end of the fifth day, after another launch of a special debug version with printf's arranged throughout the code (yes, it is on the issue "printf vs trendy cool tools") we found out the following thing. Our analyzer generates special cmd-scripts to analyze files and these scripts contain this line:

cd e:\projects\mylib

i.e. transfer to the project's folder. It was right the new version (the previous version worked well on our user's computer) where we had greatly rewritten the mechanism of generating these scripts. The cd command has one specific feature - it does not transfer into a folder on a different disk unless you specify the "/d" option. That is, the correct way of writing this should look this way:

cd /d e:\projects\mylib

This key, "/d", had been in the previous version of the program but we missed it through an oversight in the new version while rewriting the code. Unfortunately, all the tests we ran were launched from the same system disk and that is why the cd command worked well without the "/d" option.

What is more, this error occurs only when you open the project to be checked through Visual Studio (File->Open...). If you open the project by choosing the sln-file, the error will not reveal itself since the working folder coincides with that specified in the command of the cd script when you open the project this way. That is why we could not recall the error for a long time.

As everyone knows, the joy you feel when finding a complicated error is darkened by the awareness of your own folly. It was the same thing with us - the programmer simply forgot about this switch when rewriting the code.

What conclusions can we draw from this story? They are trivial yet not less useful for beginner programmers:

If your program does not work on a user computer, you should not blame programs like CUDA or Intel C++ Compiler installed on it. Although sometimes it might be their fault as well.
Even if an application successfully passes five different types of tests, it does not matter that it works in the same way everywhere and every time.
No software tool can fully replace the human brain. Alas.
If users of your program are programmers themselves, lucky you are! What other users know the words "debug version", "dump" and so on?