[lowyek]

I find it fascinating that while in other fields you see lot of theorums/results much before practical results are found. But in this forefront of innovation - I have hardly seen any paper discussing hallucinations and lowerbound/upperbound on that. Or may be I didn't open hacker news on that right day when it was published. Would love to understand the hallucination phenomena more deeply and the mathematics behind it.

[hbn]

> the hallucination phenomena

There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output. But sometimes we can tell either immediately or through fact checking it spat out a string of text that claims something incorrect.

If you start asking an LLM for political takes, you'll get very different answers from humans about which ones are "hallucinations"

[raincole]

I don't know why the narrative became "don't call it hallucination". Grantly English isn't my mother tongue so I might miss some subtlty here. If you know how LLM works, call it "hallucination" doesn't make you know less. If you don't know how LLM works, using "hallucination" doesn't make you know less either. It's just a word meaning AI gives wrong[1] answer.

People say it's "anthropomorphizing" but honestly I can't see it. The I in AI stands for intelligence, is this anthropomorphizing? L in ML? Reading and writing are clearly human activities, so is using read/write instead of input/output anthropomorphizing? How about "computer", a word once meant a human who does computing? Is there a word we can use safely without anthropomorphizing?

[1]: And please don't argue what's "wrong".

[hatthew]

LLMs model their corpus, which for most models tends to be factually correct text (or subjective text with no factuality). Sure, there exist factually incorrect statements in the corpus, but for the vast majority of incorrect statements there exist many more equivalent but correct statements. If an LLM makes a statement that is not supported by the training data (either because it doesn't exist or because the equivalent correct statement is more strongly supported), I think that's an issue with the implementation of the model. I don't think it's an intrinsic feature/flaw in what the model is modeling.

Hallucination might not be the best word, but I don't think it's a bad word. If a weather model predicted a storm when there isn't a cloud in the sky, I wouldn't have a problem with saying "the weather model had a hallucination." 50 years ago, weather models made incorrect predictions quite frequently. That's not because they weren't modeling correct weather, it's because we simply didn't yet have good models and clean data.

Fundamentally, we could fix most LLM hallucinations with better model implementations and cleaner data. In the future we will probably be able to model factuality outside of the context of human language, and that will probably be the ultimate solution for correctness in AI, but I don't think that's a fundamental requirement.

[shrimp_emoji]

It should be "confabulation", since that's not carting along the notion of false sensory input.

Humans also confabulate but not as a result of "hallucinations". They usually do it because that's actually what brains like to do, whether it's making up stories about how the world was created or, more infamously, in the case of neural disorders where the machinery's penchant for it becomes totally unmoderated and a person just spits out false information that they themselves can't realize is false. https://en.m.wikipedia.org/wiki/Confabulation

[nl]

> There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output.

This is a very "closed world" view of the phenomenon which looks at an LLM as a software component on its own.

But "hallucination" is a user experience problem, and it describes the experience very well. If you are using a code assistant and it suggests using APIs that don't exist then the word "hallucination" is entirely appropriate.

A vaguely similar analogy is the addition of the `let` and `const` keywords in JS ES6. While the behavior of `var` was "correct" as-per spec the user experience was horrible: bug prone and confusing.

[IanCal]

It's the new "serverless" and I would really like people to stop making the discussion between about the word. You know what it means, I know what it means, let's all move on.

We won't, and we'll see this constant distraction.

[sandworm101]

Hallucination is emergent. It cannot be found as a thing inside the AI systems. It is a phenomena that only exists when the output is evaluated. That makes it an accurate description. A human who has hallucinated something is not lying when they speak of something that never actually happened, nor are they making any sort of mistake in their recollection. Similarly, an AI that is hallucinating isn't doing anything incorrect and doesn't have any motivation. The hallucinated data emerges just as any other output, only to evaluated by outsiders as incorrect.

[mortenjorck]

It is an unfortunately anthropomorphizing term for a transformer simply operating as designed, but the thing it's become a vernacular shorthand for, "outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept.

There's definitely room for a better label, though. "Empirical mismatch" doesn't quite have the same ring as "hallucination," but it's probably a more accurate place to start from.

[emporas]

Chess engines, which are used for 25 years by the best human chess players daily, compute the best next move on the board. The total number of all possible chess positions is more than all the atoms in the universe.

Is is possible for a chess engine to compute the next move and be absolutely sure it is the best one? It's not, it is a statistical approximation, but still very useful.

[sqeaky]

Yet for their value as tools the truth value of statements made by LLMs do matter.

[wincy]

Well what the heck was Bing Chat doing when it wrote me a message all in emojis like it was the Zodiac killer telling me a hacker had taken it over then spitting out Python code to shutdown the system, and giving me nonsense secret messages like “PKCLDUBB”?

What am I suppose to call that?

[beernet]

How are 'hallucinations' a phenomenon? I have trouble with the term 'hallucination' and believe it sets the wrong narratuve. It suggests something negative or unexpected, which it absolutely is not. Language models aim at, as their name implies, modeling language. Not facts or anything alike. This is per design and you certainly don't have to be an AI researcher to grasp that.

That being said, people new to the field tend to believe that these models are fact machines. In fact, they are the complete opposite.

[dennisy]

Not sure if there is a great deal of maths to understand. The output of an LLM is stochastic by nature, and will read syntactical perfect, AKA a hallucination.

No real way to mathematically prove this, considering there is also no way to know if the training data also had this “hallucination” inside of it.

[ben_w]

I think mathematical proof is the wrong framework, in the same way that chemistry is the wrong framework for precisely quantifying and explaining how LSD causes humans to hallucinate (you can point to which receptors it binds with, but AFAICT not much more than that).

Investigate it with the tools of psychologically, as suited for use on a new non-human creature we've never encountered before.

[cainxinth]

Not a paper, but a startup called Vectara claimed to be investigating LLM hallucination/ confabulation rates last year:

https://www.nytimes.com/2023/11/06/technology/chatbots-hallu...

[eskibars]

FYI, I work at Vectara and can answer any questions.

For us, we treat hallucinations as the ability to accurately respond in an "open book" format for retrieval augmented generation (RAG) applications specifically. That is, given a set of information retrieved (X), does the LLM-produced summary:

1. Include any "real" information not contained in X? If "yes," it's a hallucination, even if that information is general knowledge. We see this as an important way to classify hallucinations in a RAG+summary context because enterprises have told us they don't want the LLMs "reading between the lines" to infer things. To pick an absurd/extreme case to show a point, the case of a genetic research firm, say, using CRISPR and finding they can create a purple zebra, if the retrieval system in the RAG bits says "zebras can be purple" due to their latest research, we don't want the LLM to override that knowledge with its knowledge that zebras are only ever black/white/brown. We'd treat that as a hallucination.

2. On the extreme opposite end, an easy way to avoid hallucinating would be for the LLM to say "I don't know" for everything thereby avoiding hallucinating by avoiding answering all questions. That has other obvious negative effects, so we also evaluate LLMs for their ability to answer.

We look at the factual consistency, answer rate, summary length, and some other metrics internally to focus prompt engineering, model selection, and model training: https://github.com/vectara/hallucination-leaderboard

[lowyek]

thank you for sharing this!

[amelius]

I don't see many deep theorems in the field of psychology either.