Our website uses cookies to enhance your browsing experience.
Your message has been sent. We will email you at
If you do not see the email in your inbox, please check if it is filtered to one of the following folders:
Webinar: Evaluation - 05.12
The developers of PVS-Studio analyzer regularly publish new articles about their tool (and sometimes about other analyzers as well) where they share the analysis results of various software projects produced by the analyzer and demonstrate code samples in which defects were found. Quite recently, a new product, CppCat, was released, which is a lightweight version of PVS-Studio at a low cost - compared to that of its heavier counterpart.
Unfortunately, we are no longer developing or supporting the CppCat static code analyzer. Please read here for details.
In this article, we are going to focus not on various technical details about the products, but rather the ways of their market positioning on different platforms, some facts from the company's history, and so on.
When commenting on articles about the company's products, people often ask if these products provide a support for the Linux platform. We also touched upon this question in our interview and got a detailed answer.
Our interviewee is Andrey Karpov, a CTO of PVS-Studio and CppCat projects. Andrey has also shared with us some of his ideas about programming languages, products of other companies and the issues of cooperation with these companies.
If you track the way a programmer evolves, you may notice a certain pattern to it. In the beginning of his career, a future developer studies some literature and asks for advice from more experienced fellows on forums. At some point, he makes a big step forward after getting kicked in the butt by the question, "Haven't you tried using a debugger?" Having discovered this tool - one of the many yet one of the most important ones - the developer goes on with his evolution. At the next step, it occurs to him (or he is prompted by someone) that his work can be significantly streamlined through scanning projects for bugs by a static code analyzer at the earlier development stages, rather than catching these bugs in the release product with the help of WinDbg and intricate exercises in debugging. It is at this point that our developer starts reading reviews of static analysis tools, both free and commercial, available on the market. Our article may well be treated as such a review.
I followed the link posted in Andrey's profile and got to the company's profile in the "Moi Krug" social network. What I got from their description there, the company is not a large one. I guess it is an advantage, for it usually implies less bureaucracy, tighter involvement into the development process, more creative work, and so on. Small companies do have their own charm as compared to large ones.
Andrey, tell us about your company please: how many people does your company currently employ, how many of them are directly involved in the product development, and a few words about what you find the most interesting and important to tell us.
Andrey: Yes, we are a small company, and we are of course dreaming of becoming a large and rich one. But this dream is being fulfilled very slowly. Let me put it this way. We are satisfied with what we have already achieved, but we are craving for something more. We do see prospects, but they are reluctant to come true.
But there are also some obvious advantages to this state of things: a calm working environment, no need to adjust ourselves to customers or release dates set by the marketing department. We can promptly react to our users' letters. Of course, the company may gain some fat and give home to bureaucracy with time, but at present our decisions and actions are swift. It's not a rare situation when we amuse one of our users by granting him a fix for the analyzer on the same day as he reported catching a bug.
There are currently 4 developers directly involved in development and promotion of the PVS-Studio/CppCat products. What the other staff is concerned, I can't tell for sure. They are either part-time employees, like our accountant, or freelancers. There is a nice term for this, "one and a half diggers".
As the experience of some companies shows, one of the basic reasons for their success is being focused on the products' quality and, so to say, on creating something they would like to possess themselves.
Do you utilize your own products to develop newer versions? In other words, do you use a stable version of PVS-Studio already on market to find defects in a PVS-Studio version under development? Can you give us any interesting examples when an older version of the analyzer found issues in a fresher one?
Andrey: I believe that PVS-Studio and CppCat are high-quality products. We can build and upload a fresh distribution package almost on any day. We use large set of automated tests of different levels and types. If a fixed version has passed all the tests, we know for sure that we can ship it to users.
Unfortunately, however smart and safe our internal code may be, it sometimes gets discredited by the factors of the hostile external environment. As a result, the impression of our tool gets spoiled. Though it's not our fault, it is our product that does not work as expected in such times. . There are too many examples of that, so here are just a few that occur to me right away:
But our readers want me to demonstrate some bugs PVS-Studio had found in itself. I was asked many times to write an article about it. Unfortunately, I still haven't collected a set of examples large enough to show them to the world. To be honest, I simply keep forgetting to jot them down when they are caught by incremental analysis. I just automatically check and fix them and go on working on the algorithm at hand. The next day I recall that I should have noted them down, but it's too late - I just can't remember what and where it was.
However, here are a couple of examples.
bool VirtualValueSaver::IsNotZero() const
{
if (m_decl == nullptr)
return false;
return m_decl->m_wiseType.m_pValue != nullptr &&
m_decl->m_wiseType.m_pValue->m_isPrecisionBound &&
m_decl->m_wiseType.m_pValue->m_minValue ==
m_decl->m_wiseType.m_pValue->m_minValue &&
m_decl->m_wiseType.m_pValue->m_maxValue != 0;
}PVS-Studio's warning: V501 There are identical sub-expressions 'm_decl->m_wiseType.m_pValue->m_minValue' to the left and to the right of the '==' operator. vivawalker.cpp 382 error
The fixed code:
m_decl->m_wiseType.m_pValue->m_minValue ==
m_decl->m_wiseType.m_pValue->m_maxValueOne more example:
static void FixSourceFileName(vstring &s,
const NormalizedFileOrDirectoryName &defaultname)
{
....
if (in[0] == '\\' && in[1] == '.' && in[1] == '\\')
{
in += 2;
continue;
}
....
}PVS-Studio's warning: V547 Expression is always false. Probably the '||' operator should be used here. vivabasepreprocessor.cpp 428
The fixed code:
if (in[0] == '\\' && in[1] == '.' && in[2] == '\\')Another one:
static void GetUseType(...)
{
....
if (kind == ntAssignExpr)
{
const Ptree *l = SafeSkipParentesis(pParentExpr);
const Ptree *r = SafeSkipParentesis(pParentExpr);
....
}PVS-Studio's warning: V656 Variables 'l', 'r' are initialized through the call to the same function. It's probably an error or un-optimized code. Consider inspecting the 'SafeSkipParentesis(pParentExpr)' expression. Check lines: 345, 346. functionselector.cpp 346
The fixed code:
const Ptree *l = SafeSkipParentesis(First(pParentExpr));
const Ptree *r = SafeSkipParentesis(Third(pParentExpr));The article "An Alternative to PVS-Studio at $250" presents the company's new product CppCat. It starts with mentioning that the tool was created almost from scratch.
In many products you can find a core around which the whole structure is built. This is, for example, the way antivirus software is constructed - around an antivirus engine. One and the same core is usually shared by a number of a company's products. What makes each of them different from the others is a system of various logic algorithms, interfaces and other user-oriented features (for example, the logging mechanism much spoken of) built around the same core.
So, do PVS-Studio and CppCat have a common core? Should we expect the same quality of analysis from the new product as from the old one in its target area? When there are two products based on the same core, those who use the lighter and cheaper version get some advantage over those who use the heavy and expensive one in that they can count on the same quality of bug fixing and product updates. Is that true for CppCat?
Andrey: Yes, PVS-Studio and CppCat are built on the same core. But their interfaces are very different.
The analysis capabilities of the two are very similar. We excluded from CppCat those diagnostics which proved not-so-successful and produced too much false alarms. And I'm not sure if it is at all bad. I believe that attentively checking 10 messages and fixing 5 bugs out of them is better than quickly scanning through a list of 100 warnings just to fix 2 bugs only.
Speaking of the possibility of CppCat drawing some percent of users from PVS-Studio, I find it possible but I wouldn't expect this percent to be large. Many of our users need specific options only PVS-Studio can offer - the set of 64-bit diagnostics, a modified "cyclomatic complexity" metric (which is not a popular option); some need PVS-Studio because they work with MSBuild, and so on.
Speaking of analyzing large projects with PVS-Studio and CppCat, can you give some approximate figures, say, for projects consisting of over 500-700 subprojects with the code size of XXX MBytes which can be built on a certain hardware configuration in a certain time interval?
It's true that most projects are much smaller than that, but I think developers of large software systems may find such figures interesting.
Andrey: We have never gathered such statistics, and I guess we should do it in the future. Currently I can give the following figures.
We have a set of test projects to test our products under Visual Studio 2010:
Complete analysis of this set takes 55 minutes on my computer. The total size of source codes is about 320 MBytes.
I have the following hardware configuration:
CppCat's pricing policy implies purchasing a license for some fixed price for one year only. There are some products by other companies with the annual licensing scheme too, but a company may ship a new version of the product every year at the same price.
Have you considered any ways of stimulating users through updates, new versions and other stuff like that to get them to prefer a product they will need to pay for every year to a product they could buy just once and keep using it until they decide to buy a new version?
Andrey: We don't like the idea of an "infinite" license. I'd say our ideology is similar to that of antivirus developers. We never stop teaching our analyzer to detect new bugs, supporting new features in C++ and new Visual Studio versions as well as new Windows versions.
In a sense, we sell "a service to avoid bugs". Maintaining it demands regular investments and efforts. So I find it fair to ask money for updates.
Some may argue, "I don't need any updates. I just want to buy it once and use it as long as I please". But it's silly. Isn't it queer to use an antivirus which was updated three years ago for the last time? The fresh version will definitely catch much more viruses. Just in the same way you'd catch more bugs with the latest version of our analyzer. Those who ask for an "infinite analyzer" don't really give a damn about the number of bugs in their code. And, as they don't actually need the analyzer, we prefer to ignore them altogether.
The demo version of CppCat is valid for 7 days. 7 days is often not enough for a busy developer to evaluate a product as he has to do this during the working hours and needs to make a well-weighted decision about suggesting the purchase to the management.
It is quite usual when the management team makes decisions about buying a product without discussing it with the employees (that is, the end users of the product). But sometimes it is the employees' needs and wishes that determine the management team's decision about purchasing a certain product.
Haven't you considered extending the trial period to one month and expanding your target audience to give your tool more advertising with the help of the end users who could make a purchase suggestion to their management?
Andrey: extending the trial period is highly disputable moment. You see, when you have only 7 days, you are keeping in mind that you need to try and study the tool. But with one month, you will constantly be "short of time" or "too lazy" for that. A short trial period stimulates action.
Have you ever thought of creating some online-service similarly to online-compilers to provide the ability to see just how the analyzer reacts to a certain code fragment?
Or an online-service where users could upload their test samples or even real-life code to check it without having to install the tool on their computer? With this you could implement some interesting pricing scheme like those in web-design which are numerous nowadays: for example, set certain restrictions for free analysis per project and ask for payment for unlimited access. I don't believe that many would prefer online-services to local work since most companies have intellectual property policies and they would not agree to upload their huge projects to some online-service. On the other hand, it is very likely that you would extend the coverage of small projects' developers not obsessed with that paranoia about intellectual property, and your service could become a test - and advertising - platform for the main products of your company. Besides, those developers who got interested could try the products both at work and at home as there would be no license (even a trial one) bound to their office computer's IDs.
Andrey: We don't favor any ideas related to the online service. Here is how we see it.
Imagine a developer creating applications in Visual Studio. He has to do but one simple action - run CppCat on his project. All is easy and clear - you either like what you get or do not. If you do, you will buy the tool. An online-service in this context is as useless as a fifth wheel.
Fantasies about uploading source code somewhere to check It are very difficult to fulfill, and most importantly, I don't see the reasons what to do this for. The analyzer needs a project that can be compiled. Otherwise it won't have the information about the meaning of types, preprocessor directives, and the ways macros should be expanded, etc. It means that we will need to deploy a full-blown environment somewhere, which can't be done automatically. We will have to make users upload the libraries they use in their projects and thus create a kind of a copy of their project in the web system for the analyzer to get the information about the preprocessor directives and all the stuff. So it turns that users will have to create and maintain a "twin project" somewhere in the cloud. And we will have to store all that heap somewhere and maintain it, as well.
The only way an online-analyzer could be useful is to perform experiments. But that's not a very good idea either. Programmers tend to interpret such experiments a way too superficially. I even wrote a post about this phenomenon once: "Myths about static analysis. The fifth myth - a small test program is enough to evaluate a tool". A human tends to draw a negative conclusion based on a couple of examples. That's why I'd rather want the programmers to check their real-life projects and catch bugs there.
CppCat is intended to be used under some of the MS Visual Studio versions, Windows thus being its target platform and desktop software - its target segment. There are indeed Windows versions for workstations and servers, but if you check various publications about large web-resources or a recently established cluster, you will mainly see Linux mentioned as a prevailing operating system in the server software segment. I'd say Windows success in the server software segment is similar to Linux' success in the desktop software segment. C++ is facing more and more competitors both in the field of desktop applications for Windows and in the server software area.
What do you think of making your products accessible on a Linux platform and expanding to cover the server software market? You wrote an interesting article about your attempts to sell your software to Google where the results of the Chrome browser's analysis were mentioned to demonstrate the usefulness of your tools. Any news about that? What do you think of cooperating with large companies to promote *nix versions of your products on the server software market? We already have players from Russia there: Yandex, Mail.ru, etc.
Andrey: You know, my first impulse was to write something like, "We don't have a Linux version and nor are we going to make one. Call us unskilled, or too lazy, or just stubborn fools. Whatever it is, it's ok, just stop getting at us with that question!" Then I decided that I would simply ask you to remove the questions about Linux from the interview. But my coworker said, "Why do you tell them we don't have a Linux version - here it is, I'm running it right now on a virtual machine!" And I understood that a detailed answer will be needed.
You see, we are a small and self-managed team. We don't have any support from parent companies or government grants. We may very easily go bankrupt if we start doing unreasonable things. It will be just enough to shut us down if expenses outweigh income during 3 months on end. Blowing up our infrastructure with the purpose of making a "Linux-version" may be such an unreasonable step.
Programmers are persevering in their disbelief that a compiled executable file is far from a ready software product. It sounds trivial, but people really don't notice the immense difference between the two.
We do have a build for the command-line version of the analyzer under openSUSE 13.1. It runs and even generates diagnostic warnings. But we don't have a "Linux-version". We don't have a software product of that kind and we are not going to make it in the foreseeable future.
Fortunately, Evgeniy Ryzhkov and I possess not only programming, but also management skills. Otherwise our company would have disappeared long ago. We can foresee the ways how creative enthusiasm may ruin everything.
Let's recall what our product is comprised of:
Now the most important thing. When speaking of Unix/Linux versions, everyone actually means a certain, very individual operating system. It's a crucial point indeed! A package for system X won't do any good to users of system Y - we will have to develop another "Linux-version" specifically for them. It means that we will have to develop different versions for at least five most popular systems (yet many others will be left unsatisfied).
So everything said above must be multiplied by the number of systems we'll have to support. Let it be 5 - can you imagine the amount of work now?
Expenses on the infrastructure will grow immensely. We'll need programmers. We'll even need a system administrator to take care of that large zoo of virtual machines and regression tests. We'll need to create a complex infrastructure to run regression tests on different virtual machines in turn. Besides, we'll need a bunch of computers or a powerful server to run tests so they take no more than a night.
Let's see: new programmers, system administrators, computers, servers, virtual machines, an additional cable to prevent power network burnout, renting a room for the server.
Unfortunately, 90% of programmers don't get all said above. They will just laugh, "Lol, the guy's gonna drag a new cable..."
An ordinary programmer has never had to run about searching for an electrician when things go wrong; he just gets an extra day-off. An ordinary programmer has never walked through the circles of the bureaucratic hell when needing to get a permission to drag a cable from the power substation. Well, the cable is just an example; it may be whatever else.
We know for sure that we will not be able to cope with all that in the foreseeable future.
Have you had any experience of cooperating with Russian software vendors? What about "Kaspersky Lab", "Acronis" and so on?
Andrey: No, we haven't had such an experience yet - neither with the mentioned companies nor with any others. I can't even remember anything of that kind. Well, we did have a conversation with Yandex once. I suppose it will be alright if I tell you since we didn't sign any NDA.
Shortly before IPO, Yandex decided to make its position stronger through buying up a number of startups. At least, such was our interpretation of their strategy. They willingly communicated with various teams, including us. But we didn't manage to establish long-term relations.
They asked us, "How much money and to whom do you owe?" We answered honestly, "No money to anyone. Our startup brings profit and we are doing well". They were amused and soon lost their interest in us. I can understand that. It's much easier and more profitable to buy unprofitable startups and thus attract skilled employees into your staff.
We simply offered them to purchase PVS-Studio, but they showed no interest. I heard that Yandex currently uses the Coverity static analyzer.
You said that CppCat project is a lightweight version of PVS-Studio with all the corporate-related stuff thrown away. Why didn't you take the traditional path of forming a product line consisting of the Standard, Professional and other versions of one product, each version possessing its own set of options? Instead, you released a new product with a new name. Was it a marketing trick? Sometimes companies release a new product and give it a short and catchy name, trying to get it deprived of any associations with their other products for these not to be overshadowed by the cheaper one. Was it the case with CppCat?
Andrey: We found that PVS-Studio's interface has become too complicated, overloaded and unsuitable for getting started quick and easy. Enabling/disabling different functions in different versions would only make everything worse. We decided to revise the interface completely and create a new tool. Ideally, we would like to focus entirely on promotion and mass sales of CppCat, keeping PVS-Studio at hand just for those who need a richer functionality than provided by CppCat.
What is your opinion of the Cppcheck product? Do you have any ideas of making a similar commercial tool for Linux with some traits of your own?
Andrey: Cppcheck is a fine tool to get started with the static analysis methodology. It can even cope with a non-compilable project, a diverse collection of files. You don't expect a high analysis quality in this case of course, but it is really easy to try.
I've already answered about Linux. No, we are not going to do that.
In the article "PVS-Studio. Prices Dropping" of October last year (2013), you announced an experiment of trying an individual pricing policy. What are the results of this experiment? Is it over or still running? Have you discovered any interesting pricing schemes?
Andrey: The experiment is running on. It's too early to draw any conclusions yet. I can only say that we see more communication with potential customers, but it's not clear yet if we will be able to earn more money. You see, we have a very long sales cycle; it's not unusual when half a year or even one year passes between the day a customer writes us the first letter and the day he purchases a PVS-Studio license.
That article also describes a situation when a single-person license is used by a whole development team, backed by legal arguments. I think such cheating should fade off with time. It is in a way "penny-wise economy" - just like buying one debugger for a team of a few developers. Pretty inconvenient.
Do you think that small companies inclined to such cheating will in time become more conscientious and therefore make it easier for you to sell single-person licenses?
Andrey: The trouble when companies purchase a single-person license to be used by a team is one of the reasons why we created CppCat. Earlier, in order to protect ourselves from that, we used to enhance the price. It narrowed PVS-Studio's covering to medium and large projects, while small teams and single developers were unfortunately lost.
CppCat is meant to improve that. Even the most saving team will be able to afford the purchase of our tool at $250 to check their projects. If they need to run it on more than one computer, they will have to buy more licenses for other developers.
By the way, this is the reason why we don't like the idea of a "floating" license. People argue they need it to be able to work on the project on their laptop, their home desktop computer and their office computer at once. But allowing them to do so will again lead to that old problem of one license being uncontrolledly used on several computers by different people. On the other hand, we don't feel like implementing a complex system to track when a single-person license is used simultaneously by different peaple.
After all, those who ask for floating licenses only pretend to be poor. Look at it this way. Imagine a developer who has a laptop and a home desktop computer along with the office computer. He has legal software he needs for his work installed on all of them. He does need all three for work, not just for fun. He is ready to work wherever he happens to be - at home or on travel. But all of this only make sence when you have a good income, isn't it? But no, people are still greedy to spend extra $250-$500 to buy a few licenses. It's not even serious to argue about that.
By the way, it reminds me of one story. In the earlier times, the Kaspersky antivirus (KAV) didn't use to have a 3-computer license. You had to buy a license for 2 computers plus a license for 1 computer. Or something like that - I don't remember the details. Some guy left an outraged message on the forum, "I want a license for 3 computers at once 'cause it's cheaper than the 2+1 scheme which I can't afford!" And somebody gave a reasonable reply, "If you can afford to buy THREE computers, you must have enough money to afford these licenses".
The article also describes the reason for creating the code analyzer and developing it into a commercial product. The tool is designed for C++ developers.
You can often see "revealing" articles about C++ published on the Net which set a one-sided and superficial emphasis on the C++ language's traps - looks like it is trendy to "blame C++" nowadays. For me, C++ is a favorite, one and only programming language, and I don't care if anyone finds it strange...
What do you think of C++'s prospects in your products' target area? C++, Windows, and desktop software combined make it risky. Is the game worth the candle?
Andrey: I don't have any doubts about that. The number of small and large projects written in C++ is so big there are simply no reasons why this language should be getting less popular in the next 20 years. Just look at the large Chromium project and you'll see that C++ programmers won't do without it in the near future.
Chromium was created quite recently, and the authors had had a lot of innovative languages to choose from. But since they had to use hundreds of third-party libraries written in C/C++, neither of those languages would have suited them. There is too much legacy code in C/C++. I'd say that C++ is actually facing the same, if not the worse, situation as FORTRAN or COBOL.
FORTRAN / COBOL programs are still widely used. For example, the Intel company makes more money on selling the FORTRAN compiler than on Intel C++ sales. And COBOL programmers are the most highly paid in the world.
You just don't see them mentioned on the Net nowadays, that's why they seem to have left the stage. Only those languages and tools that need promotion are intensively talked about. But there's just no reason to write about FORTRAN or COBOL - they are doing well enough.
What can you say about competing products and what are the advantages of your products over them? Have you managed to solve or avoid any of their drawbacks in your own tools? Are there any aspects to competing products which are both their strong and weak sides at the same time and which your products are clear of?
Andrey: It's not an easy subject to discuss - first of all because we can't even get a demo version from anyone :)
I think our main advantage is the easiness of obtaining a demo version and prompt technical support from us. Persuading a large company to change or add something to their tool may be an awfully difficult task. Even if you do manage to do that, your request will take months to be fulfilled. In our company, most issues can be solved in a few days.
Well, what conclusions can be drawn from all said above? The company whose products we were discussing today is a small business that produces pretty interesting software tools. The most crucial problem facing small business is the necessity to keep a strict budget account which prevents small companies from fulfilling and trying revolutionary ideas. It could be solved through the attraction of creditors, investors, etc. But you have to seriously weigh all pros and cons before making this step.
Fortunately, the company is living and making profit! And it even finds the strength to release new products on market - the CppCat analyzer in this case. There are many interesting aspects to the way this product is positioned on market, and the authors have a confident answer to the main question any business must face, "Why should a target user buy our product?" This answer is given in the article "CppCat Static Analyzer Review" which shows concrete figures in favor of their product based on the results of the product trial usage.
At the same time this tool is oriented toward Windows only. The developer team is constantly being asked questions about their intentions of making a Linux version. Perhaps Andrey and his coworkers should pay more attention to them. Sometimes customers don't have any concrete ideas about what exactly they would like a new product to look like and can only make general suggestions on improving the current products. In this case, the company may try conquering the market by releasing some new solution that will receive positive feedback from the customers. I'm viewing Andrey's counterarguments concerning Linux just from this particular viewpoint. Perhaps the team shouldn't plunge straight into committing all those vast expenses Andrey has told about but spend some money on hiring an analyst to investigate the subject and give considerations about the appropriateness of releasing a Linux version. There is a well-known rule saying that 20% of work bring in 80% of profit. Lucky or not, the world is changing and becoming more polarized, so that rule sets the 10/90 ratio now. It is possible that the analyst will confirm the company's fear that 90% of their efforts will bring in only 10% of profit. But it may also appear that releasing Linux versions of their products will be that very revolutionary idea to let the guys' dreams come true. I think that Andrey and his coworkers should think it over. Or at least make it as clear as possible to their customers that they want to focus on Windows software only and that they would rather keep a good possition supported by customers in this segment than make a leap in other direction and risk losing everything.
While reading blogs on C++, I have noticed that developers are growing more and more interested in static code analysis. This fact indicates that C++ software is growing more complex and developers' time becomes more expensive, which is a positive tendency from the viewpoint of static analysis software vendors as their tools will be widely used to perform the everyday routine of catching bugs, being a cheaper alternative to the efforts of developers, quality control department's staff and so on.
So I wish good luck to the developer team of CppCat and PVS-Studio. Many thanks to Andrey for his detailed answers. Andrey is also thankful for giving him an opportunity to collect and share many of his ideas with the readers and, in particular, show a few examples demonstrating how their products can be used to improve themselves, which the users have been asking for a long time.
Thank you for reading and thanks to Andrey for the interview!
0
0
0
0