[mofeien]

For another, more detailed take on the same topic, but with a more competent "villain", check this out: https://www.lesswrong.com/posts/KFJ2LFogYqzfGB3uX/how-ai-tak...

[adriand]

That was extremely long and I skimmed the second half (the “post breakout” portion) but the first half was very well done, and did not strike me as implausible.

[Aeolun]

Alignment gone wrong is a pretty plausible issue. I think the scary thing is a situation in which the AI also misrepresents its abilities.

Turning off the 10x human capability AI is pretty much a given. Turning off the 10x human AI that pretends to be 0.5x, not so much.