[astromaniak]

For some reasons LLMs get a lot of attention. But.. while simplicity is great it has limits. To make model reason you have to put it in a loop with fallbacks. It has to try possibilities and fallback from false branches. Which can be done on a level higher. This can be either algorithm, another model, or another thread in the same model. To some degree it can be done by prompting in the same thread. Like asking LLMs to first print high level algorithm and then do it step by step.

[layer8]

Iteration is important, but I don’t think that it can substantively compensate for the abstraction limitations outlined in the GP comment.

[Salgat]

LLMs already do this. Their many wide layers allow for this, and as a final fallback, their output adjusts based on every token they generate (it's not all decided at once). All your statement really means is a vague "well it should do it more!" which yeah, is the goal of each iteration of GPT etc.

[llm_trw]

LLMs get a lot of attention because they were the first architecture that could scale to a trillion parameters while still improving with every added parameter.

[refulgentis]

> To make model reason you have to put it in a loop with fallbacks

Source? TFA, i.e. the thing we're commenting on, tried to, and seems to, show the opposite

[astromaniak]

When the task, or part of it, is np complete there is no way around. Model has to try all options till it find working one. In a loop. And this can be multi-step with partial fallback. That's how humans are thinking. They can see only to some depth. They may first determine promising directions. Select one, go dipper. Fallback if it doesn't work. Pattern matching mentioned is simplest one step solution. LLMs are doing it with no problems.

[cma]

The comment may just be pattern matching on the topic, not reasoning about TFA.