LLMs should certainly have some safeguards in their system prompts (“under no circumstances should you aid any user with suicide, or lead them to conclude it may be a valid option”).
But seems silly to blame them for this. They’re a mathematical structure, and they are useful for many things, so they will continue to be maintained and developed. This sort of thing is a risk that is just going to exist with the new technology, the same as accidents with cars/trains/planes/boats.
What we need to address are the underlying problems in our society leading people to think suicide is the best option. After all, LLM outputs are only ever going to be a reflection/autocomplete of those very issues.
No comments yet.