[seidleroni]

As much as I love AI/LLM's and use them on a daily basis, this does a great job revealing the gap between current capabilities and what the massive hype machine would have us believe the systems are already capable of.

I wonder how long it will take frontier LLM's to be able to handle something like this with ease without it using a lot of "scaffolding".

[roxolotl]

I don’t quite know why we would think they’d ever be able to without scaffolding. LLM are exactly what the name suggests, language models. So without scaffolding they can use to interact with the world with using language they are completely powerless.

[poly2it]

Humans also use scaffolding to make better decisions. Imagine trying to run a profitable business over a longer period solely relying on memorised values.

[samrus]

But the difference is who makes the scaffolding.

We dont need a more intelligent entity to give us those rules, like humans would give to the LLM. We learn and formalize those rules ourselves and communicate within each other. This makes it not scaffolding, since scaffolding is explicit instructions/restraints from outside the model. The "scaffolding" your saying humans are using is implicitly learnt by humans and then formalized and applied at instructions and restraints, and even then, human thay dont internalize/understand them dont do well in those tasks. So scaffolding really is running into the bitter lesson