[MrCheeze]

With 2, the real problem is that approximately 0% of the OpenAI employees actually believed in the mission. Pretty much every single one of them signed the letter to the board demanding that if the company's existence ever comes into conflict with humanity's survival, the company's existence comes first.

[blagie]

That's the reality of every organization if it survives long enough.

Checks-and-balances need to be robust enough to survive bad people. Otherwise, they're not checks-and-balances.

One of the tricks is a broad range of diverse stakeholders with enforcement power. For example, if OpenAI does anything non-open, you'd like organizations FSF, CC, and similar to be represented on their board and to be able to enforce those rules in court.