[babyshake]

There is another factor to consider. The stakes of asking an AI about a taboo topic are generally considered to be very low. The number of people who have asked ChatGPT something like "how to make a nuclear bomb" should not be an indication of the number of people seriously considering doing that.

[hsbauauvhabzb]

That’s an extreme example where it’s clear to the vast majority of people asking the question that they probably do not have the means to make one. I think it’s more likely that real world actions come out of the question ‘how do I approach my neighbour about their barking dogs’ at a far higher rate. Suicide is somewhere between the two, but probably closer to the latter than the former.