To get a trial key
fill out the form below
Team License (a basic version)
Enterprise License (an extended version)
* By clicking this button you agree to our Privacy Policy statement

Request our prices
New License
License Renewal
--Select currency--
* By clicking this button you agree to our Privacy Policy statement

Free PVS-Studio license for Microsoft MVP specialists
* By clicking this button you agree to our Privacy Policy statement

To get the licence for your open-source project, please fill out this form
* By clicking this button you agree to our Privacy Policy statement

I am interested to try it on the platforms:
* By clicking this button you agree to our Privacy Policy statement

Message submitted.

Your message has been sent. We will email you at

If you haven't received our response, please do the following:
check your Spam/Junk folder and click the "Not Spam" button for our message.
This way, you won't miss messages from our team in the future.

Perl 5: How to Hide Errors in Macros

Perl 5: How to Hide Errors in Macros

Oct 05 2018

Perl 5 was chosen to expand the list of open source programming languages that have been tested using the PVS-Studio static code analyzer. This article is about found errors and difficulties when viewing analysis results. The number of macros in the code is so great that it seems that the code is written not in the C programming language, but in its peculiar dialect. In spite of the difficulties when viewing code, it was possible to collect interesting problems that will be demonstrated in this article.



Perl is a family of two high-level, general-purpose, interpreted, dynamic programming languages. Development of Perl 5 started in 1994. After a couple of decades, the code in the C programming language with many macros makes today's developers feel nervous.

Perl 5 source code was taken from the official repository (branch blead). To check the project, the PVS-Studio static code analyzer was used. The analysis was performed on the Linux operating system, but the analyzer is also available on Windows and macOS.

Viewing the analysis results was not a simple task. The fact of the matter is that the analyzer checks the preprocessed .i files, in which all preprocessor directives are already expanded, and issues warnings for source code files. This is correct behavior of the analyzer, you do not need to change anything, but many warnings are issued on macros! And unreadable code lies behind macros.

The ternary operator is working not the way you think

V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '-' operator. toke.c 9494

STATIC char *
S_scan_ident(pTHX_ char *s, char *dest, STRLEN destlen, I32 ck_uni)
  if ((s <= PL_bufend - (is_utf8)
                          ? UTF8SKIP(s)
                          : 1)
        && VALID_LEN_ONE_IDENT(s, PL_bufend, is_utf8))

Let's start an overview with a nice error. Every few code reviews I have to repeat that the ternary operator has almost the lowest priority in calculations.

Let's look at the following code fragment with an error:

s <= PL_bufend - (is_utf8) ? UTF8SKIP(s) : 1

Order of operations that a programmer expects:

  • ?:
  • -
  • <=

What is happening in reality:

  • -
  • <=
  • ?:

Here is a chart with operations priorities: "Operation priorities in C/C++".

V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '==' operator. re_exec.c 9193

S_regrepeat(pTHX_ regexp *prog, char **startposp, const regnode *p,
            regmatch_info *const reginfo, I32 max _pDEPTH)
  assert(STR_LEN(p) == reginfo->is_utf8_pat ? UTF8SKIP(STRING(p)) : 1);

Code with a similar error. Nevertheless, if you do not know the priorities of operations, you can make a mistake in the expression of any size.

Another place with an assert:

  • V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '==' operator. re_exec.c 9286

V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '&&' operator. pp_hot.c 3036

  MgBYTEPOS_set(mg, TARG, truebase, RXp_OFFS(prog)[0].end);

And here is a warning for the macro... To understand what is happening, even macro implementation will not help, because it also uses several macros!

Therefore I cite a fragment of the preprocessed file for this line of code:

(((targ)->sv_flags & 0x00000400) && (!((targ)->sv_flags & 0x00200000) ||
S_sv_only_taint_gmagic(targ)) ? (mg)->mg_len = ((prog->offs)[0].end),
(mg)->mg_flags |= 0x40 : ((mg)->mg_len = (((targ)->sv_flags & 0x20000000)
&& !__builtin_expect(((((PL_curcop)->cop_hints + 0) & 0x00000008) ?
(_Bool)1 :(_Bool)0),(0))) ? (ssize_t)Perl_utf8_length( (U8 *)(truebase),
(U8 *)(truebase)+((prog->offs)[0].end)) : (ssize_t)((prog->offs)[0].end),
(mg)->mg_flags &= ~0x40));

Somewhere here the analyzer questioned about proper use of the ternary operator (3 of them), but I have not found enough energy to get what was going on in that code. We have already seen that the developers make such errors, so it could be likely here as well.

Three more cases of using this macro:

  • V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '&&' operator. pp_ctl.c 324
  • V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '&&' operator. regexec.c 7335
  • V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '&&' operator. re_exec.c 7335

Note by a colleague Andrey Karpov. I have been meditating for 10 minutes on this code and I'm inclined to the view that there are no errors. Anyway, it's very painful to read such code, and it's better not to write this way.

Errors in Conditions

V523 The 'then' statement is equivalent to the 'else' statement. toke.c 12056

static U8 *
S_add_utf16_textfilter(pTHX_ U8 *const s, bool reversed)
  SvCUR_set(PL_linestr, 0);
  if (FILTER_READ(0, PL_linestr, 0)) {
  } else {
  PL_bufend = SvEND(PL_linestr);
  return (U8*)SvPVX(PL_linestr);

I think you can get by without inspecting the contents of macros to make sure that suspiciously duplicated code fragments take place.

V564 The '|' operator is applied to bool type value. You have probably forgotten to include parentheses or intended to use the '||' operator. op.c 11494

OP *
Perl_ck_rvconst(pTHX_ OP *o)
  gv = gv_fetchsv(kidsv,
    o->op_type == OP_RV2CV
      && o->op_private & OPpMAY_RETURN_CONSTANT
        ? GV_NOEXPAND
        : iscv | !(kid->op_private & OPpCONST_ENTERED), iscv // <=
        ? SVt_PVCV
        : o->op_type == OP_RV2SV
      ? SVt_PV
      : o->op_type == OP_RV2AV
          ? SVt_PVAV
          : o->op_type == OP_RV2HV
        ? SVt_PVHV
        : SVt_PVGV);

This code is very strange. The "iscv | !(kid->op_private & OPpCONST_ENTERED)" expression isn't used anyway. These is clearly some sort of a typo here. For example, it is possible, that this should have been written here:

: iscv = !(kid->op_private & OPpCONST_ENTERED), iscv // <=

V547 Expression 'RETVAL == 0' is always true. Typemap.c 710

  if (items != 0)
    croak_xs_usage(cv,  "");
    SysRet  RETVAL;
#line 370 "Typemap.xs"
    RETVAL = 0;
#line 706 "Typemap.c"
      SV * RETVALSV;
      RETVALSV = sv_newmortal();
      if (RETVAL != -1) {          // <=
        if (RETVAL == 0)           // <=
          sv_setpvn(RETVALSV, "0 but true", 10);
          sv_setiv(RETVALSV, (IV)RETVAL);
      ST(0) = RETVALSV;

The RETVAL variable is checked twice in a row. However, it can be seen from the code that this variable is always equal to zero. Perhaps in one or in both conditions a developer wanted to check a pointer RETVALSV, but made a typo.

Explosion of warnings about the sizeof operator

In the analyzer, there are several types of diagnostic rules, which search for bugs related to the sizeof operator usage. In the Perl 5 project, two such diagnostics summarily issued about a thousand of warnings. In this case, macros are to blame, not the analyzer.

V568 It's odd that the argument of sizeof() operator is the 'len + 1' expression. util.c 1084

char *
Perl_savepvn(pTHX_ const char *pv, I32 len)

In code there many similar macros. I chose one for example, we are interested in the argument "len + 1".

The marco is expanded by the preprocessor in the following way:

(newaddr = ((void)(__builtin_expect(((((( sizeof(size_t) < sizeof(len+1) ||
sizeof(char) > ((size_t)1 << 8*(sizeof(size_t) - sizeof(len+1)))) ?
(size_t)(len+1) : ((size_t)-1)/sizeof(char)) > ((size_t)-1)/sizeof(char))) ?
(_Bool)1 : (_Bool)0),(0)) && (S_croak_memory_wrap(),0)),

The analyzer warning is issued for the construction sizeof(len +1). The fact of the matter is that no calculations in the arguments of the operator sizeof are executed. Various macros are expanded in such code. Probably, it is the old legacy code, where nobody wants to touch anything, but current developers continue to use old macros, assuming they behave differently.

Null Pointer Dereference

V522 Dereferencing of the null pointer 'sv' might take place. pp_ctl.c 577

OP * Perl_pp_formline(void)
  SV *sv = ((void *)0);
  switch (*fpc++) {
  case 4:
    arg = *fpc++;
    f += arg;
    fieldsize = arg;
    if (mark < sp)
      sv = *++mark;
    else {
      sv = &(PL_sv_immortals[2]);
      Perl_ck_warner( (28 ), "....");
  case 5:
    const char *s = item = ((((sv)->sv_flags & (....)) == 0x00000400) ? ....

This code fragment is entirely taken from the preprocessed file, because it is impossible to make sure the problem takes place according to the source code, again because of macros.

The sv pointer is initialized by zero during declaration. The analyzer detected that, in the switch branch corresponding to the value 5, this pointer that has not been initialized before, gets dereferenced. Changing of the sv pointer takes place in the branch with the value 4 but in the end of this block, there is the operator break. Most likely, this place requires additional coding.

V595 The 'k' pointer was utilized before it was verified against nullptr. Check lines: 15919, 15920. op.c 15919

Perl_rpeep(pTHX_ OP *o)
  OP *k = o->op_next;
  U8 want = (k->op_flags & OPf_WANT);   // <=
  if (   k                              // <=
      && k->op_type == OP_KEYS
      && (   want == OPf_WANT_VOID
          || want == OPf_WANT_SCALAR)
      && !(k->op_private & OPpMAYBE_LVSUB)
      && !(k->op_flags & OPf_MOD)
  ) {

In this code fragment, the analyzer has detected a pointer k, which is dereferenced one line before it is checked for validity. This can be either an error, or redundant code.

V595 diagnostic finds many warnings in any project, Perl 5 is no exception. There is no way to pack everything in the single article, so we shall confine ourselves with one example, but developers, if they wish, will check the project themselves.

Other Issues

V779 Unreachable code detected. It is possible that an error is present. universal.c 457

  if (items != 1)
    croak_xs_usage(cv, "sv");
  else {
    SV * const sv = ST(0);
    STRLEN len;
    const char * const s = SvPV_const(sv,len);
    if (!SvUTF8(sv) || is_utf8_string((const U8*)s,len))

In the line XSRETURN_EMPTY, the analyzer has detected unreachable code. In this function, there are two return operators, and croak_xs_usage, which is a macro that expands into a noreturn function:

void Perl_croak_xs_usage(const CV *const cv, const char *const params)

In such places of the Perl 5 code, the macro NOT_REACHED is used to specify the unreachable branch.

V784 The size of the bit mask is less than the size of the first operand. This will cause the loss of higher bits. inffast.c 296

void ZLIB_INTERNAL inflate_fast(z_streamp strm, unsigned start)
  unsigned long hold;         /* local strm->hold */
  unsigned bits;              /* local strm->bits */
  hold &= (1U << bits) - 1;

The analyzer has detected a suspicious operation in code which works with bit masks. A variable of a lower size than the hold variable is used as a bitmask. This results in the loss of higher bits. Developers should pay attention to this code.



Finding errors through macros was very difficult. Viewing of the report took a lot of time and effort. Nevertheless, the article included very interesting cases related to real errors. The analyzer report is quite large, there are definitely much more exciting things. However, I cannot view it further :). I recommend developers checking the project themselves, and eliminating defects that they will be able to find.

P.S. We surely want to support this exciting project and we are ready to provide developers with a license for a few months.

Popular related articles
Free PVS-Studio for those who develops open source projects

Date: Dec 22 2018

Author: Andrey Karpov

On the New 2019 year's eve, a PVS-Studio team decided to make a nice gift for all contributors of open-source projects hosted on GitHub, GitLab or Bitbucket. They are given free usage of PVS-Studio s…
Appreciate Static Code Analysis!

Date: Oct 16 2017

Author: Andrey Karpov

I am really astonished by the capabilities of static code analysis even though I am one of the developers of PVS-Studio analyzer myself. The tool surprised me the other day as it turned out to be sma…
The way static analyzers fight against false positives, and why they do it

Date: Mar 20 2017

Author: Andrey Karpov

In my previous article I wrote that I don't like the approach of evaluating the efficiency of static analyzers with the help of synthetic tests. In that article, I give the example of a code fragment…
How PVS-Studio Proved to Be More Attentive Than Three and a Half Programmers

Date: Oct 22 2018

Author: Andrey Karpov

Just like other static analyzers, PVS-Studio often produces false positives. What you are about to read is a short story where I'll tell you how PVS-Studio proved, just one more time, to be more atte…
The Ultimate Question of Programming, Refactoring, and Everything

Date: Apr 14 2016

Author: Andrey Karpov

Yes, you've guessed correctly - the answer is "42". In this article you will find 42 recommendations about coding in C++ that can help a programmer avoid a lot of errors, save time and effort. The au…
Technologies used in the PVS-Studio code analyzer for finding bugs and potential vulnerabilities

Date: Nov 21 2018

Author: Andrey Karpov

A brief description of technologies used in the PVS-Studio tool, which let us effectively detect a large number of error patterns and potential vulnerabilities. The article describes the implementati…
PVS-Studio ROI

Date: Jan 30 2019

Author: Andrey Karpov

Occasionally, we're asked a question, what monetary value the company will receive from using PVS-Studio. We decided to draw up a response in the form of an article and provide tables, which will sho…
PVS-Studio for Java

Date: Jan 17 2019

Author: Andrey Karpov

In the seventh version of the PVS-Studio static analyzer, we added support of the Java language. It's time for a brief story of how we've started making support of the Java language, how far we've co…
The Evil within the Comparison Functions

Date: May 19 2017

Author: Andrey Karpov

Perhaps, readers remember my article titled "Last line effect". It describes a pattern I've once noticed: in most cases programmers make an error in the last line of similar text blocks. Now I want t…
The Last Line Effect

Date: May 31 2014

Author: Andrey Karpov

I have studied many errors caused by the use of the Copy-Paste method, and can assure you that programmers most often tend to make mistakes in the last fragment of a homogeneous code block. I have ne…

Comments (0)

Next comments
This website uses cookies and other technology to provide you a more personalized experience. By continuing the view of our web-pages you accept the terms of using these files. If you don't want your personal data to be processed, please, leave this site.
Learn More →