[overactor]

I think this misses the point made by Hinton. The point is that, if a model is to reach a certain level of autocomplete, it needs to understand what it is talking about. Imagine the result of what required deep thought from you is included in the training data, along with billions of other such exchanges. At some point the only way to improve training score is to develop the ability to "understand" things. (That is, if the model isn't big enough to memorize the training data.)

Universal autocomplete implies understanding and the only ending question is if our current architectures and model sizes are able to achieve any level of it.

[attilakun]

I agree, Hinton's point is more about next-token-prediction being a good training objective for understanding, and not whether LLMs reproduce every aspect of a human's thought processes. They clearly don't (at least not at present scale), otherwise we'd have AGI now!

[hopfenspergerj]

I agree. We should prompt the model with the statement of the Riemann hypothesis. If the autocomplete is good, the model will output a proof.

[overactor]

No human can solve the Riemann Hypothesis. Why do you expect an AI to do it before you consider it able to understand things?

[dboreham]

> it needs to understand what it is talking about

"to understand" is a concept invented by the human brain to help it operate. It doesn't actually mean anything concrete in the physical world. See: the Chinese room argument (paradoxically it proves the converse of what Searle was arguing -- that there is no such thing as understanding, because you can emulate a system with identical outputs from the same set of inputs without caring or labeling what happens inside the room).

[scudsworth]

that's, uh, not the conclusion of the chinese room argument

[ImHereToVote]

The Chinese room doesn't have a model; it has an infinite dictionary. A sufficiently large LLM isn't a dictionary, it has to contain a model of some sort.