[quirkot]

I always wonder... if there was an AGI and it's chipset gave the wrong answer, how would it ever know?

[GuB-42]

The neural networks we use today have really terrible accuracy, and we tend to make them worse, not better, as having more neurons is better than having more precision. Human brains are also a mess, but somehow, they work, and we are usually able to correct our own mistakes.

Since by AGI, we usually mean human-like, that system should be able to self correct the same way we do.

[nenenejej]

How do humans know? usually someone corrects someone else. we have repeatability in physics, or we wait 30 years and quash convictions etc. etc.

[gs17]

I'd presume it could reason around the wrong answer, at least to realize something was off. Current LLMs will sometimes hallucinate that this has happened when they're "thinking".