[dekervin]

It's way worse than that. First, We interact with LLMs through private conversation and we are used to have private conversation with human we trust. Some of that trust will be transfered to LLMs. Second, LLMs have a vastly bigger "mental" power to build a long term mental model of us, while we interact with them. Which mean they can chose with extreme precision their words to trigger an emotion, a certain reaction.

Combine the two and the potential for manipulation, suggestion, preference altering is through the roof.

[AyyEye]

The next step is to combine it with heartrate/bloodpressure/eye tracking in phones and generate the text you're reading in realtime based on biofeedback. We'll be able to control people like robots. See where those $1MM+ salaries and billions of dollars are going, yet?