[ClumsyPilot]

The frustrating thing about your argument is that it runs on a pretence that we must prove squares aren’t circles.

A person may be unable to provide mathematical proof and yes be obviously correct.

The totally obvious thing you are missing is that most people will not encourage obviously self-destructive behaviour because they are not psychopaths. And they can get another person to intervene if necessary

Chatbots do not have such concerts

[BeetleB]

I'm not sure I get the actual point you're making.

To begin with, not all therapy involves people at risk of harming themselves. Easily over 95% of people who can benefit from therapy are at no more risk of harming themselves than the average person. Were a therapy chatbot to suggest something like it to them, the response will either be amusement or annoyance ("why am I wasting time on this?")

Arguments from extremes (outliers) are the stuff of logical fallacies.

As many keep pointing out, there are plenty of cases of licensed therapists causing harm. Most of the time it is unintentional, but for sure there are those who knowingly abused their position and took advantage of their patients. I'd love to see a study comparing the two ratios to see whether the human therapist or the LLM fare worse.

I think most commenters here need to engage with real therapists more, so they can get a reality check on the field.

I know therapists. I've been to some. I took a course from a seasoned therapist who also was a professor and had trained them. You know the whole replication crisis in psychology? Licensed therapy is no different. There's very little real science backing most of it (even the professor admitted it).

Sure, there are some great therapists out there. The norm is barely better than you or I. Again, no exaggeration.

So if the state of the art improves, and we then have a study showing some LLM therapists are better than the average licensed human one, I for one will not think it a great achievement.