Let's Play a Game – find bugs in popular open-source projects

[ViktorasM]

so frustrating when the answer is obvious but where to click is not.

e.g. in "if a.length != a.length" the "correct" token is the first "a". Should really be anything in the whole expression.

[__david__]

Yeah I fell victim to that, too. Also having a timer counting down caused me to freeze up. I got about half the questions right immediately, and half after the time had ticked down so that my mind could relax and pay attention. I think I legitimately only got 2 wrong. And one of them I could argue that I found a different bug than the one they were looking for.

[kartD]

Completely agree, I got three wrong, but because I clicked the wrong spot, I only got 3 right, it's pretty frustrating.

The test is also pretty easy. I only know C and some Java and I figured most of the problems. The standout important questions to me were the memset and free ones.

[cocoflunchy]

The game: http://q.viva64.com/

[Odin78]

Do similar games exist for other languages? I like the concept but my C++ is rusty at best.

[ucho]

It is easier to find a bug then to select token that is "correct". Also some of those bugs are detected even by GCC.

[unwind]

So why are they still "live" in the originating projects, then? Honest question. Perhaps nobody reads their compiler warnings?

[iheartmemcache]

I'd bet a weeks salary (seriously) that Coverity (yeah yeah it's expensive, whatever) will catch > 90% of these with maybe a 10% false positive, and Code Analysis in VS would hover around the 80% mark with maybe a 15% false positive (both for C++; for ANSI C, move that up to 95% and 90% accordingly).

ccc-analyzer/c++-analyzer for clang/llvm would probably hit the 90-ish mark for ANSI C too.

[Sylos]

Do I get extra points for finding an exploit in their game?

Just disable JavaScript and reload the page, which will halt the countdown and display the solution. Then, when you've read what you need to know, re-enable JavaScript, reload the page, again, and then just click the place where the error is. Then click on the "Next Question"-button, disable JavaScript, again, and so on...

[yoo1I]

> Do I get extra points for finding an exploit in their game?

The highest "badge" that you can earn by completing the 15 questions is <trollface> "won by tricking the system".

So: yes.

[junke]

    .right-fragment {
        color: green;
        font-weight: bold;
        background-color: white;
        font-size: 30px;
    }

[Kenji]

Wow, I found that surprisingly difficult. I noticed that when I read code, I optimize my attention for bugs _I_ make, which are often of a different nature than the bugs that this static analyzer finds. Clearly, I have to work on my bug awareness.

[zbyszek]

I imagine that the conclusion Mr Karpov would have you reach is that you should buy PVS Studio instead.

[joebergeron]

Interesting, but some of the questions are somewhat shallow or ambiguous. For example, referencing a variable x0 where y0 should have been used, etc.

Although I do have to say, it's certainly kind of alarming, if not amusing, to see trivial bugs that slipped through the cracks in very many massively popular pieces of software.

[kbart]

The hardest part for me was to read an ugly formatted code with no highlights.

[Superleroy]

Agree, badly formatted and thus hard to read. And when you find the error you don't know where to click as it only highlights a part of the error.

[dagurp]

And what kind of sadist names a variable l1?

[chatmasta]

That's a luxury! Try finding a bug in an obfuscated binary, without access to the source code or even header files, that implements anti-debugger techniques to crash or behave differently upon first detection of a debugger.

[SixSigma]

That's an argument against highlights.

The Plan9 community has to defend no highlights to n00bs.

[delinka]

Many of the bugs the quiz wants me to find would be pointed out by my IDE (but I'm not taking the quiz in my IDE...) Also, lots of this code is terrible. Simply adhering to a reasonable style could highlight or eliminate bugs. For example, "variable > other_value - 1"; let's get that "- 1" outta there and use >= instead.

Anyway, although I suggest these would be helped by a modern IDE and some style guides, I'd concede that since this is open source code, it's probably older than modern IDEs...

[thomasahle]

The whole point of this is that static analysis can find bugs that people have missed. If your IDE finds most of these, it just means that its static analysis is mostly as good as the Intel one.

[the_af]

"This code is terrible" is not a good argument. Yes, it's buggy code. However, we know most programmers write bugs. This is an argument in favor of static analysis (and yes, your IDE is a form of that).

[pjmlp]

I guess you never looked at typical enterprise code.

Also even on 2016, many live on the AT&T world where IDEs are not welcome.

[cloudjacker]

> Many of the bugs the quiz wants me to find would be pointed out by my IDE

No no, you are supposed to pretend like you might be in a situation where this is no IDE!

Tech company interviews in a nutshell

[bluetomcat]

These are mostly mistypings caused by carelessness which occasionally play oddly with C/C++ syntax, not the kind of logical bugs which plague most software.

[the_af]

I don't understand. These are real bugs from actual software. Not sure if the test is randomized; I got stuff like a != a and multiplications by (y - y). Also C++-specific stuff like improper use of sizeof and references. How are these not the kind of logical bugs that plague most software?

[garaetjjte]

Some of these errors will be propably detected by compiler with -Wall and -Wextra.

[wepple]

that's what I thought. > 50% of the questions I was presented with were misprint errors which likely would've been caught by unused variable warnings.

[alblue]

From the intro to the game:

"Test does not support mobile devices. It is very easy to miss with finger. We are working on new version of tests with better mobile devices support, new problems to solve etc. However, it is not implemented yet."

Since you have to select the bug in the code with the mouse, it's unusable on touchscreen mobile devices.

[bjourne]

I got 13 points (not counting as incorrect those where I knew the answer, but the system wouldn't accept the token I clicked on) and I don't think it was that hard. My objection is that if you run static analysis on code like this:

  static int rr_cmp(uchar *a,uchar *b)
  {
    if (a[0] != b[0])
      return (int) a[0] - (int) b[0];
    if (a[1] != b[1])
      return (int) a[1] - (int) b[1];
    if (a[2] != b[2])
      return (int) a[2] - (int) b[2];
    if (a[3] != b[3])
      return (int) a[3] - (int) b[3];
    if (a[4] != b[4])
      return (int) a[4] - (int) b[4];
    if (a[5] != b[5])
      return (int) a[1] - (int) b[5];
    if (a[6] != b[6])
      return (int) a[6] - (int) b[6];
    return (int) a[7] - (int) b[7];
  }

You are doing it wrong. You first need to figure out how to write code in a more readable fashion and not allow ugly things like the above in the codebase.

[thomasahle]

This looks like legacy code from before compilers had good loop unrolling. If people have large code bases like these to maintain, they'll probably be happy to find the bugs now, and worry about rewriting it all later.

[esmi]

I wouldn't be surprised if this code is actually auto generated from a macro, higher level script, or something. I do driver work and this kind of thing is very common in that space.

[ryanjodonnell]

It's like Pokemon Go, except you never leave the house and search through code instead of the real world. Perfect.

[Retr0spectrum]

10/15. I guess that's not bad for someone who never used C++ before.

This test gives the user a slightly unfair advantage, because you know where to look. If I was given the whole of the source file for any of these examples, I would probably never have found any bugs.

[tener]

It is good to be reminded of the horrors of C++. I'm happy to be working with C# and have Resharper enabled. It easily catches all of the bugs in this test.

[AndreyKarpov]

C#... huh :)

Accord.Net - http://www.viva64.com/en/b/0410/

Microsoft WPF Examples - http://www.viva64.com/en/b/0407/

Xamarin.Forms - http://www.viva64.com/en/b/0400/

Lucene.Net - http://www.viva64.com/en/b/0381/

Xenko Game Engine - http://www.viva64.com/en/b/0379/

Space Engineers - http://www.viva64.com/en/b/0376/

WPF examples by the Infragistics Company - http://www.viva64.com/en/b/0375/

CNTK tool kit - http://www.viva64.com/en/b/0372/

Sony C#/.NET component - http://www.viva64.com/en/b/0371/

IronPython and IronRuby - http://www.viva64.com/en/b/0367/

MonoDevelop - http://www.viva64.com/en/b/0366/

CoreFX - http://www.viva64.com/en/b/0365/

Roslyn - http://www.viva64.com/en/b/0363/

[drzaiusapelord]

Scary stuff here. My second question was a fairly obvious bug in OpenSSL.

[tudorw]

got 3 and proud of it, signed an [ex]php dev ;)

(well, 1, and 2 where my spidey sense tingled but I clicked the wrong place)

[xyproto]

The only way to win is not to play.