I develop software for embedded systems (mostly for STM32 and Milandr). As the main IDE, I use uVision Keil. As long as I write in C and C++, I've been wondering whether I write code the right way. Can I write it that way?
This article was published on habr.com. It was copied and translated with the author's permission.
Well, my code does compile. But it's C++, where "program is ill-formed, no diagnostic required" is ok.
For several years, I've been bothering my higher-ups asking to buy a PVS-Studio license. Unexpectedly my request coincided with an urgent need to spend money on new software. Finally, we bought a PVS-Studio license!
On the one hand, I was overjoyed. But on the other hand, I faced some difficulties integrating PVS-Studio into Keil. PVS-Studio is integrated into Visual Studio (this pleased our department that develops for desktops) and JetBrains products (CLion, Rider, Idea, Android Studio) right away. PVS-Studio also provides ready-to-use scripts to integrate into some other build systems. As for Keil, PVS-Studio supports only compilers. So, we needed someone to integrate PVS-Studio into Keil. Who's going to do that? Well, since that was my idea...
Obviously, I still had to do my usual tasks, so the integration process was put off. At first, regardless of all recommendations, I was checking projects "only on holidays", without any automation. I used the universal scenario – run PVS-Studio Standalone, click "Start compiler monitoring", compile the project, and view the analysis results.
I've been using this scenario until one day I spent 3 days debugging a very unpleasant bug. The bug appearances were random and bewildering. It turned out to be a banal null pointer dereference (which on microcontrollers usually does not result in any instantaneous errors like Access Violation).
I quickly realized that PVS-Studio detects this bug. That was the final nail in the coffin of my patience! – and started integrating PVS-Studio into Keil.
Now let me explain what I mean by integration:
At the end of the article, you'll know that almost everything we needed, we made happen – but with some changes :)
As far as I know, Keil does not provide any "usual" ways of customization such as plugins or extensions. So, the only way to integrate into the build is Custom Build Steps, which are called "User Scripts" in Keil.
In project options, there's the Users tab that allows running third-party programs (only .bat or .exe, there's no even .cmd!) for three events:
The first and the last events seem to be enough. The plan looks simple:
Quick experiments showed that Build Output (as expected) catches all the output in stout and stderr for user scripts. Although Build Output doesn't show the Cyrillic alphabet at all, that's why errors in these scripts turn into unreadable scrawls. I used a duct tape and replaced the code page with an English one, so the errors were issued in English.
Okay, let's point out the main steps.
Luckily (or maybe PVS-Studio developers did it on purpose), warning lines format in PVS-Studio is the same as in Keil. That's why, I managed to jump to the line with error by double-clicking.
So, is that the end of the story?
Unfortunately, no.
After a while, I noticed a strange thing – I'm rebuilding the same project without any changes, but the results of the PVS-Studio analysis are different! An error kept appearing and disappearing in one of the files.
An epic e-mail thread with technical support began. It lasted for almost a year (!), however, it was totally my fault. Honestly, PVS-Studio has the best technical support I've ever met. Believe me, I contacted many technical supports – from Russian chip makers, who congratulated me on "Raspberry Jam Pie Day" (no, I'm not joking) to the largest foreign companies, that kept sending me from one person to another for months :)
I must confess that I replied less frequently than PVS-Studio support... because I still had my job duties. However, it only partially justifies me.
Anyway, the problem was clear – there is no magic wand to monitor compiler runs. If the compiler rapidly compiled a file, PVS-Studio might skip its run. Undoubtedly, "rapidly" is a relative term. The compile time depends on environment variables, the number of background processes, and so on. Apparently, the key element is running build in parallel. If the parallel build is enabled, PVS-Studio is highly likely to skip a run. If the parallel build is disabled, PVS-Studio does not skip runs. At least, working with several projects, I didn't notice such behavior on my computers.
Okay. What should I do about it?
The brute-force method is to disable the parallel build (at least, sometimes; to run the analysis). It's a bad choice, because:
Let's move on. Soon I realized that it was silly to use monitoring. The project file contains all the necessary information – which files are compiled, which keys are used, and so on. Why don't we just parse this file?
This solution looks good in name only. It's not clear, who should do this parsing. Indeed, we bought the license, but we can't endlessly exploit PVS-Studio tech support. For them, our Keil issue is not a high priority one. Its integration requires lots of efforts and time. PVS-Studio deals with many client's requests, therefore it is unprofitable to integrate into every environment. That is why PVS-Studio suggests a universal monitoring solution.
Besides, although the project is in xml format, it is closed. So, a vendor can introduce significant and unpredictable changes at any time.
Also, as I understand, it's not enough to analyze the information just in the project file.
Keil provides a weird feature – creating a project batch file. I still don't know the purpose of this feature. This batch file contains all the necessary information for PVS-Studio, and it's enabled with a single check mark!
Unfortunately, this check mark also breaks the incremental build. That is, any compilation becomes a complete recompilation. It affects the build time, so, unfortunately, it's not an option for us.
If monitoring can't catch compiler run, let's just make compiler run longer!
The ‑‑preinclude flag forcibly included the template calculation in each cpp-file in the project.
I didn't use these solutions because they slow down compilation (and also because it's cringey).
Eventually, we have two options left. They are not perfect. Both options have pros and cons. But, as they say, the perfect is the enemy of the good.
The first option is not to monitor the compilation every time we run the analysis. It is enough to get a set of compiled files. This set is rarely changed – only when new files are added to the project (or when old files are removed).
Thus, this option has two stages:
How to detect changes in the list of files? Certainly, there are different ways. The first idea that came to my mind was to use git since all projects must be gitted.
If the project file has been changed since the last commit, the project file contains new files!
Many things may be changed in the project file because it contains compilation options and lots of other stuff. So, I wrote the following line:
was_changed=$(git diff *.uvproj* | grep "[+,-]\s*<FileName>" \
| sed -e 's#</*FileName>##g')Do you remember, I said earlier that it's better not to parse a closed and undocumented format? Well, forget it :D
Or we can just monitor all changes in the project file without delving into details; thus, we'll have more false-positive warnings, but not false-negative ones.
Okay, we realized that the set of files has changed – how to start monitoring?
Here I have not come up with anything better than to issue a warning to the user and ask to do the following:
The disadvantages of this approach:
Some less interesting details:
To collect a dump, we use the following path:
CLMonitor.exe save Dump -d "path_to_dump\pvs_dump.zip"When we already have the dump, the analysis starts as follows:
CLMonitor.exe analyzeFromDump -d "path_to_dump\pvs_dump.zip"
-l "path_to_result\pvs.plog"
-t "path_to_config\pvs_settings.xml"
-c "path_to_config\ignore_warnings.pvsconfig"
PlogConverter.exe "path_to_result\pvs.plog" --renderTypes=Txt
-o "path_to_result"
more "path_to_result\pvs.plog.txt"pvs_settings.xml and ignore_warnings.pvsconfig configs allow you to suppress warnings. I'll explain it in detail later.
The purpose of these actions is to obtain the result from the dump, render it into text, and output a text file to the terminal. As I mentioned, the output format is the same as Keil expects. So, double-clicking on a warning works :)
It's inconvenient to do everything manually. After I discussed possible solutions with the PVS-Studio team, they developed a special utility and several scripts for us.
Here's the main idea:
The main script is long. I won't copy-paste it here (but here's the link to it on github); besides, the PVS-Studio team offered me the script :) I changed it a little and removed the necessity to manually specify the path to the Keil folder.
The calls in this case look as follows:
Here "Target 1" is the name of your current target, it must be quoted
Letters with hashes – called "Key Sequence" in Keil — are build variables, environment variables for the build.
The advantages of this approach compared to the previous one:
Cons:
This is a disadvantage only in comparison with the previous approach, where we don't need to specify the name :) Fortunately, you have to do it only once, when creating the configuration, but you have to do it manually.
I couldn't remove these messages :(
Since Keil does not provide usual "Errors" window, as in most other IDEs, you have to constantly read the Build Output window. It is impossible to filter these messages. These messages clutter compiler output and make it hard to find compilation errors and warnings.
Luckily, we don't have to run the script each time before the compilation of each file. We run it only if the set of compiled files has changed. But we again must select and deselect checkboxes manually! It looks like the previous approach—we still have to select and deselect checkboxes but in one place, not in two.
So, this integration is not perfect, but it's better than nothing.
Since we discuss integration, I should mention different ways to suppress warnings. You can find all the necessary information on the PVS-Studio website. Here, I'll try to briefly describe the suppression mechanism. I'll skip some options since I don't use them.
So, you can suppress warnings on several levels:
|
How the warnings are suppressed |
When to use |
|---|---|
|
Suppress one specific warning for one specific line |
If you are sure, it's not an error |
|
Suppress all warnings for a folder in the current project |
If we have a library in the project folder |
|
Suppress a class of warnings for the current project |
If this analysis class doesn't work anyway |
|
Suppress specific types of warnings in the current project |
For warnings that usually do not correspond to actual errors, but are issued constantly |
|
Suppress specific folders on the computer |
For constantly used libraries outside the project folder |
Embedded systems help you to make each project self-sufficient (i.e., the project does not depend on any external libraries, you just clone it, and it compiles). I wanted to keep this self-sufficient approach. Therefore, all scripts for analysis and files for warnings suppression must be stored in the project folder (obviously, you have to install PVS-Studio separately).
Suppress one specific warning for one specific line
Just write a comment before the certain line or on the right of it. The comment looks as follows:
// -V::XXX – A message explaining why this warning is suppressedHere XXX is the warning number.
I think it's crucial to write the explanation; otherwise, it's impossible to understand why the warning is suppressed. Is it suppressed because the warning is false or because it annoys the programmer, who could not understand the issue?
Suppress all warnings for a folder only in the current project
Here an xml file is used (I usually name it pvs_settings.xml). This file travels with the project.
The example:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<!-- Import settings (mostly exclude paths) from global settings -->
<AutoSettingsImport>true</AutoSettingsImport>
<PathMasks>
<!-- Exclude this paths from analysis -->
<string>\cmsis\</string>
<string>\spl\</string>
</PathMasks>
<!-- Disable 64-bit errors -->
<Disable64BitAnalysis>true</Disable64BitAnalysis>
</ApplicationSettings>Suppress a class of warnings for the current project
Here the ignore_warnings.pvsconfig file is used. This file travels with the project too. Of course, we welcome messages explaining why the warning is ignored!
The example:
###### Common warnings
# ignore 64-bit warnings
// -V::4
# allow C-style cast for primitive integer types (and void)
// -V:int:2005
// -V:char:2005
// -V:short:2005
// -V:uint8_t:2005
// -V:int8_t:2005
// -V:uint16_t:2005
// -V:int16_t:2005
// -V:uint32_t:2005
// -V:int32_t:2005
// -V:uint64_t:2005
// -V:int64_t:2005
// -V:void:2005
# ignore 'The body of the statement should be enclosed in braces';
# that doesn't look like a source of errors for us
// -V::2507
###### MISRA
# ignore MISRA C++ 6-6-5
# 'A function should have a single point of exit at the end.'
# this goes againts our best practises and generally seems outdated
// -V::2506Suppress specific folders on the computer
It is done with the help of xml files in the current user's folder. The local file must contain the <AutoSettingsImport>true</AutoSettingsImport> line, showing PVS-Studio that they must be used with the local xml file. PVS-Studio looks through the %APPDATA%\PVS-Studio\SettingsImports folder and applies all the files.
The example:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationSettings xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<PathMasks>
<!-- Exclude this paths from analysis -->
<string>\boost\</string>
<string>\zlib\</string>
<string>\png\</string>
<string>\libpng\</string>
<string>\pnglib\</string>
<string>\freetype\</string>
<string>\ImageMagick\</string>
<string>\jpeglib\</string>
<string>\libxml\</string>
<string>\libxslt\</string>
<string>\tifflib\</string>
<string>\wxWidgets\</string>
<string>\libtiff\</string>
<string>\mesa\</string>
<string>\cximage\</string>
<string>\bzip2\</string>
</PathMasks>
</ApplicationSettings>It's possible to integrate PVS-Studio into Keil, however all the solutions are not perfect and require some manual enhancements.
I've been using PVS-Studio for several years now. I am satisfied because I feel protected from my own stupidity :)
I must admit that it's difficult to evaluate the benefits of constant analysis because any error is fixed almost immediately after the analyzer issues the corresponding warning. It's complicated to estimate the time I would have spent to find this issue without PVS-Studio.
It's worth noting that the analyzer slows down the build, so sometimes I disable it – for example, during furious debugging when I have to constantly change some coefficient in a single line.
There are some questions that I should have asked myself before starting the integration:
Here are some examples on github.