top | item 34456613

(no title)

wikfwikf | 3 years ago

One of the amazing things about Hypothesis is that it's very good at generating weird inputs that trigger bugs in cases when it would be impossible to test more than a tiny fraction of all possible inputs.

There is very very clever code that does things like create weird dicts where the keys are weird strings and the values are other weird dicts or arrays or strings..

If it finds a test failure, it then applies a reduction step, where it replaces the extremal test case with something slightly more benign and checks if the test still fails. This allows it to generate a test case which is just hairy enough to trigger the bug, but no more. This makes it easier to understand why that specific test case fails.

discuss

order

PartiallyTyped|3 years ago

Oh so it’s like QuickTest in Erlang and Haskell. Very interesting, thanks!

Does anyone have any recommendations for papers on this?

wikfwikf|3 years ago

I believe it was heavily inspired by QuickTest.