top | item 33657884

(no title)

marwatk | 3 years ago

I've read the assertions section a few times and I still don't understand the argument. How is:

  if got == nil {
    t.Errorf("blog post was nil, want not-nil")
  }
Better than

  assert.NotNil(t, got, "blog post")
? They seem to suggest that you lose context, but their "Good" examples are similarly devoid of context.

discuss

order

Beltalowda|3 years ago

The main reason I dislike the assert libraries is that you lose the ability to format things nicely; being able to see what's going on quickly is pretty nice when tests fail. Sometimes you want %s, sometimes %q, sometimes maybe a diff or formatted as something else. Testify "solves" this by just dumping a lot of stuff to your terminal, which is a crude brute-force solution. Plus with table-driven test you're saving what, 2 or 3 lines of code?

For example for a simple string comparison it's 12 lines:

   --- FAIL: TestA (0.00s)
       --- FAIL: TestA/some_message (0.00s)
           main_test.go:11:
                   Error Trace:    /tmp/go/main_test.go:11
                   Error:          Not equal:
                                   expected: "as\"d"
                                   actual  : "a\"sf"
   
                                   Diff:
                                   --- Expected
                                   +++ Actual
                                   @@ -1 +1 @@
                                   -as"d
                                   +a"sf
                   Test:           TestA/some_message
 
vs. 2 lines with a "normal" test:

   --- FAIL: TestA (0.00s)
       --- FAIL: TestA/some_message (0.00s)
           main_test.go:11:
               have: "as\"d"
               want: "a\"sf"
With your NotNil() example it's 4 lines, which seems about 3 lines more than needed.

This kind of stuff really adds up if you have maybe 3 or 4 test failures.

jrockway|3 years ago

NotNil is fine, but as a library author you need to implement every comparison operator for every type. You also have to implement each assertion twice; once for t.Error and once for t.Fatal. We have a homegrown assertion library in our codebase at work, and every time I write a test I have to settle for an inaccurate assertion, or add two more methods to the library. I grew up on Go at Google where I would not have been allowed to check in assertion helpers, and I think they made the right call there.

I've seen a lot of gore in code that uses assertion libraries like assert.Equals(t, int64(math.Round(got)), int64(42)). Consider the error message in that case when got is NaN or Inf.

It is so easy to write your own assertions. I can type them in my sleep and in seconds:

    if got, want := f(), 42; got != want { 
        t.Errorf("f:\n  got: %v\n want: %v", got, want)
    }

    if got, want := g(), 42.123; math.Abs(got - want) > 0.0001 {
        t.Fatalf("g:\n  got: %v\n want: %v", got, want)
    }
Why implement a NotEquals function when != is built into the language?

(The most popular assertion library also inverts the order of got and want, breaking the stylistic convention. It's so bad. Beware of libraries from people that wrote them as their first Go project after switching to Go from Java. There are a lot of them out there, and they are popular, and they are bad.)

Finally, cmp.Diff is the way to go for complex comparisons. Most use of assertion libraries can be replaced by cmp.Diff. I wouldn't use it for simple primitive type equality, but for complex data structures, it's great. And very configurable, so you never have to "settle" for too much strictness or looseness.

dgunay|3 years ago

Drawing inspiration from:

> Complex assertion functions often do not provide useful failure messages and context that exists within the test function.

I think the best compromise is to avoid combining individual logical units into one assertion. For example:

Bad:

  assert.True(t, myStr == "expected" && myInt == 5)
Good:

  assert.Equal(t, "expected", myStr)
  assert.Equal(t, 5, myInt)
Real world example: at my workplace we have some code that tests HTTP request formation for correctness (right URL, body, headers, etc). Replacing big all-or-nothing booleans with individual assertions on each property of the request provides much more useful test failure messages.

As with any published "best practices" like this, have an open mind but don't just cargo cult whatever Google does. Best to be selective about what does and doesn't work for your situation.

adamwk|3 years ago

Coming from Swift development I first missed having a collection of assertion functions for testing, but I've come around to the go testing patterns. I do think test assertion libraries usually result in less useful messages, and you end up having to implement a new function for every type of comparison under the sun.

(One thing I absolutely abhor is those assertion DSLs similar to rspec)