[ceres]

Okay this is kinda random and maybe off topic but can someone please explain?

When I tell an LLM to count to 10 with a 2 second pause between each count all it does is generate Python code with a sleep function. Why is that?

A 3 year old can understand that question and follow those instructions. An LLM doesn’t have an innate understanding of time it seems.

Can we really call it AGI if that’s the case?

That’s just one example.

[schoen]

It seems right that LLMs don't have an innate understanding of time, although you could analogize what you did with writing someone a letter and saying "please count to ten with a two-second pause between numbers". When you get a letter back in the mail, it presumably won't contain any visible pauses either.

[pegasus]

That's because you used a LLM trained to produce text, but you asked it to produce actions, not just text. An agentic model would be able to do it, precisely by running that Python code. Someone could argue that a 3 year old does exactly that (produces a plan, then executes it). But these models have deeper issues of lack of comprehension and logical consistency, which prevents us (thankfully) from being able to completely remove the necessity of a man-in-the-middle who keeps an eye on things.

[throwawaymaths]

just because it doesn't do what you tell it to doesn't mean it's not intelligent. i would say doing something that gets you where you want when it knows? it can't do exactly what you asked for (because architecurally it's impossible) could be a sign of pretty intelligent sideways thinking!!? dare i say it displays a level of self awareness that i would not have expected.