Go Fuzz Testing

[throwawaylala1]

That first example is deceptive. In this line:

    for i := 0; i <= n; i++ {

The "<=" character just looks like a "<" character due to the typeface that the author chose. I'm going to assume that wasn't an accident because it perfectly masks the obvious bug in the for loop.

When I did a quick scan of that code it looked good to me. Then he claimed it should panic and I felt crazy. So I copy/pasted the code into an editor to see wtf was going on and sure enough it was "<=".

[Andoryuuta]

This is my main issue with fonts made for programming that use special ligature marks. They _look_ cool, but they are very hard to tell apart from other characters.

It's like someone decided that the I, l, 1 confusion wasn't enough and decided to add more. :P

[lalaithion]

    go test fuzz v1
    string("000000000000000000000000000000Ö00000000000000000000000000000")
    rune('\u0083')
    int(60)

Interesting that the minimization engine wasn't able to shrink this further.

[evmunro]

I noticed that as well - most fuzzers will have a maximum duration or number of iterations they're allowed to attempt when minimizing so as not to starve out actual inputs. It could be that the fuzzer hit that limit, or potentially prioritizes readable inputs over small inputs.

[jiehong]

It’s a bit surprising to have fuzzing as part of the standard tool chain, but not property based testing.

I guess we still need to rely on something like gopter.

[masklinn]

> It’s a bit surprising to have fuzzing as part of the standard tool chain, but not property based testing.

Property testing doesn’t really benefit from (let alone need) toolchain integration, so while having better support for proptesting would be nice and give them more visibility, it’s far from critical.

[akshayshah]

testing/quick is in the standard lib, though it's a very far cry from Hypothesis or QuickCheck.

[kardianos]

Use table driven tests.

func TestFoo(t *testing.T) { list := []struct {

     want string
     input string
     expectedErr error
  }{
  {item1},
  {item2},
  }

  runtest

[spencerwgreene]

I'm pretty sure fmt.Println("HELLO", str, len(str), len(result), n) isn't supposed to be in the final solution? :)

[andrei]

Good catch :) fixed!

[rendall]

Neat, Andrei. Good luck with this venture!

[Thaxll]

Curious if fuzzing is a common thing in network programming, does fuzzing plays nicely with binary inputs?

[andrei]

It does! Fuzzing actually started off as a tool built by security researchers to find vulnerabilities in parsers, and other complex codebases, usually written in C/C++ (looking for memory bugs). So anything that deals with untrusted binary data is a prime candidate for fuzz testing.

Go’s fuzzing framework supports `[]byte` arguments as well as all of the standard Go primitives, so you should be able to test netcode this way.

If you're looking for a C/C++ solution, my recommendation is libfuzzer [0]. We've also built our own C/C++ fuzzing engine at Fuzzbuzz [1].

[0] https://llvm.org/docs/LibFuzzer.html

[1] https://docs.fuzzbuzz.io/docs/getting-started-in-c-or-c++

[masklinn]

Along side file parsers it’s a pretty major fuzzing target, as it tends to be exposed to malicious inputs.

Fuzzing works primarily on binary data, “structured” fuzzing is somewhat rarer.

[monkpit]

What’s the rationale behind bundling a fuzzing library in the language instead of as a separate library?

[masklinn]

To do guided fuzzing (graybox, like afl or libfuzzer) you need to instrument the binary. To instrument the binary, you need to be part of the building process.

Since Go has a bespoke compilation toolchains and AFAIK doesn’t have compiler plugins, external fuzzing tools had to either fork the toolchain or perform extensive pre and post processing (couldn’t tell you what go-fuzz did but many article about go-fuzz note that the building process can take a while).

As such, building fuzzing into the standard toolchain and maintaining it as part of the project makes a lot of sense. It also gives fuzzing a much higher level of visibility (because sadly there will always be a population for whom an external / third-party tool will be suspicious).

[LukeShu]

It does instrumented fuzzing. The older https://github.com/dvyukov/go-fuzz would rewrite your sources to inject the instrumentation and pass the rewritten sources to the compiler, but it didn't really work with Go modules. This is something that probably makes sense to integrate with the compiler toolchain, same as `go test`'s coverage testing.

[spicybright]

Probably to encourage a standard fuzzing library for the language. Like the same way go fmt is part of the language.