[sawert]

I think people also get hung up on this: at some level, we too are just predicting the next 'token' (i.e., taking in inputs, running them through our world model, producing outputs). Though we're obviously extremely multimodal and there's an emotional component that modulates our inputs/outputs.

Not arguing that the current models are anywhere near us w/r/t complexity, but I think the dismissive "it's just predicting strings" remarks I hear are missing the forest for the trees. It's clear the models are constructing rudimentary text (and now audio and visual) based models of the world.

And this is coming from someone with a deep amount of skepticism of most of the value that will be produced from this current AI hype cycle.