[natbennett]

I’ve worked on a large project that got more reliable after they removed the QA team. Removing the QA team was incidental to the quality improvement though — the QA team got removed because basically it was just producing a nightly report of problems that didn’t matter.

What actually improved quality on that project was, basically, architecture improvements. A major subsystem was rewritten in a way that made large classes of important bugs impossible. That rewrite was substantially supported by automated tests, though — a rather advanced simulation system IIRC.

I do agree that “automated tests” aren’t especially good at finding bugs. For that there’s really no replacement for human beings who care looking carefully at the system’s behavior, whatever title those people have. They’re mostly useful when they make it safer to make changes to the code’s design.

[Nathanba]

yes I think the way we use the word "bug" generally means that it's a bug from a human being's point of view. But automated tests are automated, maybe what they catch shouldn't even be called bugs but just type/check/assumption errors. Then the expectation is far more aligned with reality.. just because a unit test doesn't run successfully doesn't mean anything. You simply have to check the assumption and adjust the type check perhaps.