[tidenly]

I wonder to what extent is hallucination a result of a "must answer" bias?

When sampling data all over the internet, your data set only represents people who did write, did respond to questions - with no representation of what they didn't. Add into that confidently wrong people - people who respond to questions on, say, StackOverflow, even if they're wrong, and suddenly you have a data set that prefers replying bullshit, because there's no data for the people who didnt know the answer and wrote nothing.

Inherently there's no representation in the datasets of "I don't know" null values.

LLMs are forced to reply, in contrast, so they "bullshit" a response that sounds right even though not answering or saying you don't know would be more appropriate - because no-one does that on the internet.

I always assumed this was a big factor, but am I completely off the mark?