You clearly have not worked on a codebase with thousands of tests. At my previous job the build system had an option to run a test N times concurrently in the cloud. I used this whenever I wanted to commit to some other project but some of their tests were garbage (to prove that test is flaky, and therefore to be ignored). You could even binary search (running 1000 times on each pivot point) to see who introduced the flakiness. Expensive but gets the job done.In my projects I either fix the nondeterminism or delete such tests.
AstralStorm|6 years ago
These are not exactly nondeterministic but sometimes people end up with that instead of pseudorandom ones.