[ozim]

Key part is *where reliability matters*, there are not that many cases where it matters.

We tell stories of Therac 25 but 90% of software out there doesn’t kill people. Annoys people and wastes time yes, but reliability doesn’t matter as much.

E-mail, internet and networking, operations on floating point numbers are only kind of somewhat reliable. No one is saying they will not use email because it might not be delivered.

[abustamam]

10% is still quite a lot!

Reliability matters in lots of areas that aren't war. Ignoring obvious ones like medicine/healthcare and driving, I want my banking app to be reliable. If they charge me $100 instead of $1 because their LLM didn't realize their currency was stored in floating point dollars and not cents, then I may not die but I'd be pretty upset!

[ozim]

I was writing about Therac 25 that’s not war that’s medical equipment and code written by a human that killed people. Without LLM.

[cobbal]

We guarantee 5 nines of uptime, and 1 nine of not killing people

[iugtmkbdfil834]

<< 90% of software out there doesn’t kill people.

As we give more and more autonomy to agents, that % may change. Just yesterday I was looking at hexapods and the first thing it tells you ( with a disclaimer its for competitions only ) that it has a lot of space for weapon install. I had to briefly look at the website to make sure I did not accidentally click on some satirical link.

[ozim]

Main point is that there is many more lines of code of CRUD business apps running on AWS and instances of applications than even non-autonomous car software even though we do have lots of cars.

[wussboy]

Most code will not kill people, but a lot of code could kill a business.