[evanextreme]

I disagree, because the television was never talking back to you. By the medium directly engaging with the person, the responsibility / enablement of this issue is more on the company than previous non interactive mediums. It would be like if a movie caused psychosis in people, and the sequel doubled down / enabled that. Even though there's no person (besides the consumer) in the loop, there is responsibility to train these systems to reduce the amount of these cases.

[pixl97]

There are people that send the vast majority of the money they make to televangelists who are not speaking to them 'directly' and yet those individuals would tell you said speaker is communicating directly with them.

This is how a lot of propaganda over the radio and TV works.

[evanextreme]

Great point, actually. In many respects these language models doing this could be similar to how YouTube used to platform Alex Jones before removing him. Regardless, I still believe its the responsibility of the creators of these models to work on mitigations

[_dark_matter_]

Counterpoint: https://en.m.wikipedia.org/wiki/Truman_Show_delusion

[im3w1l]

A key question I think is whether these models can be made safer for mentally ill without cutting into utility for the healthy. Is the danger inherent to this technology, or is it incidental?

And here I think we are fortunate that there doesn't seem to be tradeoff.

[evanextreme]

Agreed here. My personal opinion is that this is fixable, but it would require reinforcement into the models that counteract the sycophantic stickiness that most companies use to drive engagement, so it won't really ever be a priority.

[visarga]

> By the medium directly engaging with the person

and

> Even though there's no person in the loop

contradict each other. There is always a person in the loop, and the LLM is actually reacting to their messages, however wrong it turns out. They could have chosen a positive interaction instead. The LLM reflects back what the human puts in.

[evanextreme]

Sorry, should have clarified. By "no person in the loop" I meant from the side of the language model itself. The person is interacting with a thing which produces content for it, and the only person engaged in that process is the person consuming the content.