[Lionga]

The LLM has an internal "confidence score" but that has NOTHING to do with how correct the answer is, only with how often the same words came together in training data.

E.g. getting two r's in strawberry could very well have a very high "confidence score" while a random but rare correct fact might have a very well a very low one.

In short: LLM have no concept, or even desire to produce of truth

[sharperguy]

Still, it might be interesting information to have access to, as someone running the model? Normally we are reading the output trying to build an intuition for the kinds of patterns it outputs when it's hallucinating vs creating something that happens to align with reality. Adding in this could just help with that even when it isn't always correlated to reality itself.

[alexwebb2]

Huge leap there in your conclusion. Looks like you’re hand-waving away the entire phenomenon of emergent properties.

[amelius]

> In short: LLM have no concept, or even desire to produce of truth

They do produce true statements most of the time, though.

[jaen]

That's just because true statements are more likely to occur in their training corpus.