Update on Analysis Results for CPython and Ruby

11 Aoû 2016

Author: Pavel Belikov

Previous statistics
Details on the new check
Error density
Conclusion

In one of our recent articles, we were comparing error density of the classical implementations of the languages Python and Ruby and made a mistake in the article itself: in the list of errors found in CPython we included errors from third-party libraries. So, we had to recheck the projects and collect the statistics anew.

Previous statistics

Some of the errors (fragments No. 2, No. 4, No. 5, No. 6) described in the previous article refer to OpenSSL. These errors affect, one way or another, every project that uses the library, including CPython, but it's incorrect to treat them as errors of the project itself. Naturally, PVS-Studio provides a mechanism for ignoring such warnings: you just need to add the directory with your externals or 3rd_party to the list of exceptions, and you won't get warnings for those directories and files anymore while they are on that list.

Details on the new check

This time we used the Linux-version of PVS-Studio, which is currently under development, to check CPython and Ruby. It works in a very simple way: you just start the build from PVS-Studio and wait until it's over. PVS-Studio tracks every compiler call, so it doesn't depend on the way the project is built. After the analysis is complete, the analyzer generates a log file.

Error density

No new errors worth mentioning were found this time. There are some errors that have to do with incorrect handling of alarm conditions: errors of this type are often found even in well tested projects because programmers don't write many tests for such situations. Here is an example of such an error found in CPython:

int
_PyState_AddModule(PyObject* module, struct PyModuleDef* def)
{
  PyInterpreterState *state;
  if (def->m_slots) {
    ....
    return -1;
  }
  state = GET_INTERP_STATE();
  if (!def)
    return -1;
  ....
}

PVS-Studio diagnostic message: V595 The 'def' pointer was utilized before it was verified against nullptr. Check lines: 286, 292. pystate.c 286

Some code fragments are just poorly formatted. There are also conditional expressions that look strange after macro expansion: for example one condition is checked several times. Generally speaking, however, everything is fine.

As for the new statistics, here they are:

The picture very much resembles the previous one except that it no longer includes errors from third-party libraries and an entire class of errors that refer to Windows-only code.

Conclusion

Few errors were found in the projects during the second check, but it's no surprise: the most critical fragments have been discussed just recently. However, static analysis should be used not only for quality control, but at the coding stage as well: finding and fixing errors at this stage is easier and cheaper.

Welcome to try PVS-Studio and apply for beta testing of the Linux-version.

#Cpp