[lewdwig]

There are nascent signs of emergent world models in current LLMs, the problem is that they decohere very quickly due to them lacking any kind of hierarchical long term memory.

A lot of what is structurally important the model knows about your code gets lost whenever the context gets compressed.

Solving this problem will mark the next big leap in agentic coding I think.