top | item 45917082

(no title)

SquareWheel | 3 months ago

Forcing shorter answers will definitely reduce their quality. Every token an LLM generates is like a little bit of extra thinking time. Sometimes it needs to work up to an answer. If you end a response too quickly, such as by demanding one-word answers, it's much more likely to produce hallucinations.

discuss

order

profunctor|3 months ago

Is this proven?

holbrad|3 months ago

I know Andrej Karpathy mentions it in his youtube series so there's a good chance of it being true.

abustamam|3 months ago

It's certainly true anecdotally. I've seen it personally plenty of times and I've seen it reported plenty of times.