[no_op]

I think Moravec's Paradox is often misapplied when considering LLMs vs. robotics. It's true that formal reasoning over unambiguous problem representations is easy and computationally cheap. Lisp machines were already doing this sort of thing in the '70s. But the kind of commonsense reasoning over ambiguous natural language that LLMs can do is not easy or computationally cheap. Many early AI researchers thought it would be — that it would just require a bit of elaboration on the formal reasoning stuff — but this was totally wrong.

So, it doesn't make sense to say that what LLMs do is Moravec-easy, and therefore can't be extrapolated to predict near-term progress on Moravec-hard problems like robotics. What LLMs do is, in fact, Moravec-hard. And we should expect that if we've got enough compute to make major progress on one Moravec-hard problem, there's a good chance we're closing in on having enough to make major progress on others.

[lsy]

Leaving aside the lack of consensus around whether LLMs actually succeed in commonsense reasoning, this seems a little bit like saying “Actually, the first 90% of our project took an enormous amount of time, so it must be ‘Pareto-hard’. And thus the last 10% is well within reach!” That is, that Pareto and Moravec were in fact just wrong, and thing A and thing B are equivalently hard.

Keeping the paradox would more logically bring you to the conclusion that LLMs’ massive computational needs and limited capacities imply a commensurately greater, mind-bogglingly large computational requirement for physical aptitude.

[nopinsight]

It's far from obvious that thought space is much less complex than physical space. Natural language covers emotional, psychological, social, and abstract concepts that are orthogonal to physical aptitude.

While the linguistic representation of thought space may be discrete and appear simpler (even the latter is arguable), the underlying phenomena are not.

Current LLMs are terrific in many ways but pale in comparison to great authors in capturing deep, nuanced human experience.

As a related point, for AI to truly understand humans, it will likely need to process videos, social interactions, and other forms of data beyond language alone.

[bjornsing]

Good points. Came here to say pretty much the same.

Moravec's Paradox is certainly interesting and correct if you limit its scope (as you say). But it feels intuitively wrong to me to make any claims about the relative computational demands of sensi-motor control and abstract thinking before we’ve really solved either problem.

Looking e.g. at the recent progress in solving ARC-AGI my impression is that abstract thought could have incredible computational demands. IIRC they had to throw approximately $10k of compute at o3 before it reached human performance. Now compare how cognitively challenging ARC-AGI is to e.g. designing or reorganizing a Tesla gigafactory.

With that said I do agree that our culture tends to value simple office work over skillful practical work. Hopefully the progress in AI/ML will soon correct that wrong.

[RaftPeople]

Also agree and also came here to say the same.