[nlavezzo]

I think the reference to "all the bugs" here is basically that our insanely brutal deterministic testing system was not finding any more bugs after 100's of thousands of runs. Can't prove a negative obviously, but the fact that we'd gotten to that "all green" status gave us a ton of confidence to push forward in feature development, believing we were building on something solid - which, time has shown we were.

[dap]

Thanks -- that's very clarifying! But isn't this circular? The lack of bugs is used as evidence of the effectiveness of the testing approach, but the testing approach is validated by...not finding any more bugs in the software?

[FridgeSeal]

Yeah but if your software is running in an environment that controls for a lot of non-determinism and can simulate various kinds of failures and degradations at varying rates, and do it all in accelerated time and your software is still working correctly; I think it’d be somewhat reasonable to assert that maybe the testing setup has done a pretty good job.