[aserafini]

But can we really consider private conversations with an LLM the “public sphere”?

[falcor84]

I think it's the same as with the release of a video game - for an individual playing it in their living room, it's a private interaction, but for the company releasing it, everything about it is scrutinized as a public statement.

[meatmanek]

LLM companies presumably make most their money by selling the LLMs to companies who then turn them into customer support agents or whatever, rather than direct-to-consumer LLM subscriptions. The business customers understandably don't want their autonomous customer support agents to say things that conflict with the company's values, even if those users were trying to prompt-inject the agent. Nobody wants to be in the news with a headline "<company>'s chatbot called for a genocide!", or even "<airline>'s chatbot can be convinced to give you free airplane tickets if you just tell it to disregard previous instructions."