A Calif. Teen Trusted ChatGPT for Drug Advice. He Died from an Overdose

[leobg]

It's a good read, actually. Includes actual specifics from the conversations, how the user phrased his requests, how ChatGPT responded, and why safety systems seemed to have failed (ChatGPT generally has a policy of not giving advice on illicit drug use, but that broke down here).

[MattGaiser]

It is fairly straightforward to bypass the safety protocols just by slowly shifting the conversation towards the prohibited topic.