[arcticfox]

The 2006 book 'Daemon' is a fascinating/terrifying look at this type of malicious AI. Basically, a rogue AI starts taking over humanity not through any real genius (in fact, the book's AI is significantly weaker than frontier LLMs), but rather leveraging a huge amount of $$$ as bootstrapping capital and then carrot-and-sticking humanity into submission.

A pretty simple inner loop of flywheeling the leverage of blackmail, money, and violence is all it will take. This is essentially what organized crime already does already in failed states, but with AI there's no real retaliation that society at large can take once things go sufficiently wrong.

[Cyphase]

I love Daemon/FreedomTM.[0] Gotta clarify a bit, even though it's just fiction. It wasn't a rogue AI; it was specifically designed by a famous video game developer to implement his general vision of how the world should operate, activated upon news of his death (a cron job was monitoring news websites for keywords).

The book called it a "narrow AI"; it was based on AI(s) from his games, just treating Earth as the game world, and recruiting humans for physical and mental work, with loyalty and honesty enforced by fMRI scans.

For another great fictional portrayal of AI, see Person of Interest[1]; it starts as a crime procedural with an AI-flavored twist, and ended up being considered by many critics the best sci-fi show on broadcast TV.

[0] https://en.wikipedia.org/wiki/Daemon_(novel)

[1] https://en.wikipedia.org/wiki/Person_of_Interest_(TV_series)

[mapt]

It was a benevolent AI takeover. It just required some robo-motorcycles with scythe blades to deal with obstacles.

Like the AI in "Friendship is Optimal", which aims to (and this was very carefully considered) 'Satisfy humanity's values through friendship and ponies in a consensual manner.'

[wink]

I liked Daemon and completely missed Freedom. Thanks for the pointer.

[throwup238]

Makes on wonder whether it will be Google, OpenAi, or Anthropic to build the first Samaritan (though I’m betting on Palantir)

[Schmerika]

> A pretty simple inner loop of flywheeling the leverage of blackmail, money, and violence is all it will take. This is essentially what organized crime already does already in failed states

[Western states giving each other sidelong glances...]

[bbrake]

PR firms are going to need to have a playbook when an AI decides to start blogging or making virtual content about a company. And what if other AIs latched on to that and started collaborating to neg on a company?

Could you imagine 'negative AI sentiment' and those same AI assistants that manage sales of stock (cause OpenClaw is connected to everything) starts selling a companies stock.

[icedchai]

I really enjoyed that book. I didn't think we'd get there so quickly, but I guess we'll find out soon enough...

[dboreham]

Is this not what has already happened over the past 10-15 years?