To get a trial key
fill out the form below
Team license
Enterprise license
** By clicking this button you agree to our Privacy Policy statement

Request our prices
New License
License Renewal
--Select currency--
USD
EUR
RUB
* By clicking this button you agree to our Privacy Policy statement

Free PVS-Studio license for Microsoft MVP specialists
** By clicking this button you agree to our Privacy Policy statement

To get the licence for your open-source project, please fill out this form
** By clicking this button you agree to our Privacy Policy statement

I am interested to try it on the platforms:
** By clicking this button you agree to our Privacy Policy statement

Message submitted.

Your message has been sent. We will email you at


If you haven't received our response, please do the following:
check your Spam/Junk folder and click the "Not Spam" button for our message.
This way, you won't miss messages from our team in the future.

>
>
>
The feedback for our C++ quiz and why i…

The feedback for our C++ quiz and why it matters

Aug 31 2022

Earlier we wrote about our quiz for C++ developers. Since the quiz launch, we have been carefully collecting your feedback. Some of the comments were related to the quiz errors, that we obviously sought to fix.

0985_fix_the_quiz/image1.png

For those who don't know yet what quiz we are talking about, let us briefly explain. We made a quiz so you could challenge yourself in code review. For this quiz, we've picked errors detected by our analyzer in Open Source projects. You'll see 10 random code fragments, each containing an error. In 60 seconds, you need to spot the bug and click on it.

You can read more about our quiz in the note, or finally try to spot the error!

But let's get back to the topic. We received a lot of comments from people who took our quiz. And some of the replies pointed out problems with the mechanics of the quiz. We, the advocates of code quality, always strive to enhance our own projects as well.

That's why we dug into the comments everywhere we posted our quiz. In social networks, on Habr, at thematic forums and websites. As a result, we identified two issues to be fixed. The first one got a lot of comments — people complained about the way code errors are selected in the quiz and the way answers taken as correct/incorrect. The second issue was not as common, since we got a single comment on it. And, ironically, we have knowingly made this issue possible. But first things first. Let's discuss.

The first issue. How to hit the correct answer?

Our quiz originally allowed to select multiple code elements for an answer. In the code fragment below, the error is clip->GetSequence.

0985_fix_the_quiz/image2.png

Previously, you could select both the correct answer, and (either deliberately or accidentally) select the symbols = or (), and even the entire line as well. As a result, you could find an actual error, but an answer can be regarded as incorrect due to extra symbols being selected.

Now you can select no more than one element in the code fragment. But that doesn't make the selected element to be the only correct. At the same time, a correct answer sometimes includes several elements of code to be selected, that's why an answer would be correct if you select any of them.

You can easily see the difference in the videos attached:

Before

After

We recently summed up the results of our quiz for two months and told you that the average score of correct answers among nearly 2,000 developers is 3.5 out of 10. But we can't deny that the way errors were counted may have affected the final results of the quiz.

The second issue. Our small fail

It's always embarrassing to admit your own mistakes, but we have to. Let's go back to the time when the quiz was created. At first, the quiz contained a bug: as soon as the question appeared on the screen, you got the "time's up" message. This was the reason:

  • the timer reset to zero when the task was loading;
  • after that the timer started from 60 seconds to 0. Every second the timer was checked: if timer === 0, then we put the "time is up" message, but since the timer reset to zero at the beginning, the null check was done immediately.

In other words:

the task is opened -> the timer resets to zero -> the check for === 0 is done and the "time is up" message is issued.

Instead, it was supposed to work this way:

the task is opened -> the timer resets to zero -> the timer starts -> after 60 seconds the check for === 0 is done and the "time is up" message is issued.

So as not to waste time, we decided to set the timer from 60 to 1. After that, the time reset to zero, and the "time is up" message was issued when checking "if the timer === 1".

The solution was not elegant, but obviously fast. And we were sure that no one would probably cosplay 90s action movie heroes, pressing the "Answer" button at the last second. But, if developers believe that everything would work exactly the way they made it, there is bound to be someone who "stumbles on a kludge".

For 2 months we safely forgot about this mistake. Until we saw a screenshot in one of the comments:

0985_fix_the_quiz/image3.png

It turned out that even though we promised the quiz participants 60 seconds for each answer, we actually gave them only 59 seconds. Well, we pleaded guilty and corrected the mistake. So, now you can safely press the "Answer" button at the last second. But it always better not to waste time, particularly when you have already found an answer.

This case has proven the essential role of feedback in any project.

In fact, we covered all these issues in advance while making the C# quiz. You can already challenge yourself in it. Try it yourself, share the quiz with your C# teammates, and of course, share your own results with us! And if you suddenly notice a bug, don't hesitate to write us – we will fix it.

Also, don't forget that those who passed the quiz get a promo code for a 30-day PVS-Studio license.

Popular related articles
Static code analyzer vs developers. Here we go again.

Date: Jul 11 2022

Author: Alexey Sarkisov

In mid-May this year we released an updated version of our quiz for C++ developers. It's been 2 months already — keep on reading to learn more about the results!
Solutions to bug-finding challenges offered by the PVS-Studio team at conferences in 2018-2019

Date: Nov 18 2019

Author: Sergey Khrenov

Hi! Though the 2019 conference season is not over yet, we'd like to talk about the bug-finding challenges we offered to visitors at our booth during the past conferences. Starting with the fall of 20…
How developers were checking projects for bugs using PVS-Studio

Date: Dec 04 2017

Author: Phillip Khandeliants

Pinguem.ru and the PVS-Studio team recently held a contest where programmers were to use PVS-Studio static analyzer for one month to find and fix bugs in the source code of open-source projects. Thei…
A talk with Jason Turner: the history of CppCast, and why it was shut down

Date: Sep 27 2022

Author: Uliana Grishina

In this article, we are going to talk to Jason Turner, one of the CppCast founders. CppCast is the first podcast for C++ developers by C++ developers. From 2015 to 2022 CppCast had weekly conversatio…
Reworking C and C++ front-end — or how we deal with 16-year legacy code in PVS-Studio

Date: Sep 22 2022

Author: Sergey Larin

In 2022, the PVS-Studio static analyzer for C and C++ turns 16 years old. If the analyzer were a human, it would have already finished high school! This project was started a long time ago, and since…

Comments (0)

Next comments
Unicorn with delicious cookie
Our website uses cookies to enhance your browsing experience. Would you like to learn more?
Accept