[sn0wleppard]

Nice place to cut the quote there

> [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

[muzani]

Yup, one of the huge flaws I saw in GPT-5 is it will constantly say things like "I have to stop you here. I can't do what you're requesting. However, I can roleplay or help you with research with that. Would you like to do that?"

[kouteiheika]

It's not a flaw. It's a tradeoff. There are valid uses for models which are uncensored and will do whatever you ask of them, and there are valid uses for models which are censored and will refuse anything remotely controversial.

[rsynnott]

Nudge nudge, wink wink.

(I am curious if this in intended, or an artefact of training; the crooked lawyer who prompts a criminal client to speak in hypotheticals is a fairly common fiction trope.)

[NuclearPM]

How is that a flaw?

[andrepd]

At the heart of this is the irresponsible marketing, by companies and acolytes, of these tools as some kind of superintelligence imbued with insights and feelings rather than the dumb pattern matching chatbots they are. This is what's responsible for giving laypeople the false impression that they're talking to a quasi-person (of superhuman intelligence at that).

[scotty79]

> these tools as some kind of superintelligence imbued with insights and feelings

What are examples of OpenAI marketing ChatGPT in this manner?

[llmthrow0827]

Incredible. ChatGPT is a black box includes a suicide instruction and encouragement bot. OpenAI should be treated as a company that has created such and let it into the hands of children.

[Spooky23]

That’s what happens when you steal any written content available without limit. In their pursuit of vacuuming up all content, I’m sure they pulled some psycho Reddits and forums with people fetishizing suicide.

[behringer]

Oh won't somebody please think of the children?!

[dgfitz]

Imagine, if instead a cop had handed the kid a gun, there would be riots in the street.

And I fucking hate cops.

[stavros]

Ah, I misread that and thought that's what the user said.