[nonfamous]

Great link, thanks for sharing. This point below stood out to me — put another way, “fixing” a system in response to an incident to make it safer might actually be making it less safe.

>>> Views of ‘cause’ limit the effectiveness of defenses against future events.

>>> Post-accident remedies for “human error” are usually predicated on obstructing activities that can “cause” accidents. These end-of-the-chain measures do little to reduce the likelihood of further accidents. In fact that likelihood of an identical accident is already extraordinarily low because the pattern of latent failures changes constantly. Instead of increasing safety, post-accident remedies usually increase the coupling and complexity of the system. This increases the potential number of latent failures and also makes the detection and blocking of accident trajectories more difficult.

[albert_e]

But that sounds like an assertion without evidence and underestimates the competence of everyone involved in designing and maintaining these complex systems.

For example, take airline safety -- are we to believe based on the quoted assertion that every airline accident and resulting remedy that mitigated the causes have made air travel LESS safe? That sounds objectively, demonstrably false.

Truly complex systems like ecosystems and climate might qualify for this assertion where humans have interfered, often with best intentions, but caused unexpected effects that maybe beyond human capacity control.

[nonfamous]

Airline safety is a special case I think — THE NTSB does incredible work, and their recommendations are always designed to improve total safety, not just reduce the likelihood of a specific failure.

But I can think of lots of examples where the response to an unfortunate, but very rare, incident can make us less safe overall. The response to rare vaccine side effects comes immediately to mind.