The dangerous comfort of AI answers

[maxvij]

The widespread trust in AI chatbots as all-knowing oracles is a dangerous misunderstanding of what they actually are. Large Language Models are sophisticated pattern-matching tools, not truth machines, a critical distinction defining how we should use them.

[bell-cot]

From a "safe or dangerous?" PoV, AI's are better understood as delusionally-overconfident sycophants. They'll happily make crap up, or twist the truth beyond recognition, in their statements to you.

[nosioptar]

I've tried using AI for factual questions pretty frequently this year.

I've yet to have a chat where it gave an accurate answer beyond a very superficial level.