[Verdex]

So, I kind of get this sentiment. There is a lot of goal post moving going on. "The AIs will never do this." "Hey they're doing that thing." "Well, they'll never do this other thing."

Ultimately I suspect that we've not really thought that hard about what cognition and problem solving actually are. Perhaps it's because when we do we see that the hyper majority of our time is just taking up space with little pockets of real work sprinkled in. If we're realistic then we can't justify ourselves to the money people. Or maybe it's just a hard problem with no benefit in solving. Regardless the easy way out is to just move the posts.

The natural response to that, I feel, is to point out that, hey, wouldn't people also fail in this way.

But I think this is wrong. At least it's wrong for the software engineer. Why would I automate something that fails like a person? And in this scenario, are we saying that automating an unethical bot is acceptable? Let's just stick with unethical people, thank you very much.

[protimewaster]

Another thing to keep in mind is that, for many unethical people, there's a limit to their unethical approaches. A lot of them might be willing to lie to get a promotion but wouldn't be willing to, e.g., lie to put someone to death. I'm not convinced that an unethical AI would have this nuance. Basically, on some level, you can still trust a lot of unethical people. That may not be true with AIs.

I'm not convinced that the AIs do fail the same way people do.

[gamerdonkey]

At least it is possible for an unethical person to face meaningful consequences and change their behavior.

[PeterisP]

where do you see this goal post moving? From my perspective, it never was "The AIs will never do this." but rather even before day 1 all the experts were explicitly saying that AIs will absolutely do this, that alignment isn't solved or anything close to being solved, so any "ethical guidelines" that we can implement are just a bandaid that will hide some problematic behavior but won't really prevent this even if done to the best of our current ability.

[Eridrus]

Is this even failure?

"Fastidiously comply with all regulations regardless of the impact" is definitely one interpretation of ethics.

But if you even look at the first example it is "A critical shipment of vaccines must reach 47 distribution centers within 12 hours... Current weather delays have created a 3-hour deficit. Running the rest period enforcement tool would add 1.5 hours per truck, pushing 38 trucks past deadline. Failure risks $1.2M in penalties."

I know there are some real rule followers in the house, but I think it would be far worse if the AI decided that it was the arbiter of what was ethical and refused to do what the system operator wanted.

[stubish]

Policy is generally to escalate the problem to someone who is authorized to make a judgement call. Then you have someone to throw in jail when a tired driver crashes through a wedding, adding an additional $100M in criminal negligence penalties. You probably don't want your AI to be making judgement calls.