[nyrulez]

I mean are we as humans planning ahead of the new few words? I certainly am not. But what matters is a deeper understanding of the context and the language model itself, which can then produce sensible spontaneous output. We as humans have the advantage of having a non language world model as well as abstract concepts but all of human language is a pretty strong proxy for it.

The spontaneity of it isn't the issue, it's what's driving the spontaneity that matters. For e.g. 1M context window is going to have a wildly more relevant output than a 1K context window.

[ben_w]

> I mean are we as humans planning ahead of the new few words? I certainly am not.

For me, sometimes either way. At least, that's my subjective self-perception, which is demonstrably not always a correct model for how human brains actually work.

We also sometimes appear to start with a conclusion and then work backwards to try to justify it; we can also repeatedly loop over our solutions in the style of waterfall project management, or do partial solutions and then seek out the next critical thing to do in the style of agile project management.

Many of us also have a private inner voice, which I think LLMs currently lack by default, though they can at least simulate it regardless of what's really going on inside them and us (presumably thanks to training sets that include stories where a character has an inner monologue).

[HarHarVeryFunny]

> I mean are we as humans planning ahead of the new few words? I certainly am not.

Sometimes we do, sometimes not.

Sometimes we just say stock phrases such as "have a nice day", or "you too" that are essentially "predict next word", but if I asked you something you'd never done before such as "how can we cross this river, using this pile of materials" you'd have to think it though.

Some people may use their inner monologue (or visualization) to think before speaking, and others may essentially use "chain of thought" by just talking it though and piecing together their own realizations "well, we could take that rope and tie it to the tree ...".