[dtheodor]

Your run of the mill computer program also "operates in enormous parameter spaces that are impossible to meaningfully test for all possible adversarial inputs and degraded outputs".

[varjag]

This is hardly similar as the state of a typical computer program can be meaningfully inspected, allowing both useful insights for adversarial test setups and designing comprehensive formal tests.

[dtheodor]

Right, if you consider the internal state, it is hardly similar. You talked about black box and QA though. Black box by definition holds the internal state as irrelevant, and QA mostly treats the software it tests as a black box, or in other words the tests are "superficial" as you call it.