Is Marble's definition of a "world model" the same as Yann LeCun's definition of a world model? And is that the same as Genie's definition of a world model?
Pretty sure it's used as a marketing term here. They train on images that you generate/give it, but the output of that training is not a model, it's a static 3d scene made up out of gaussian splats. You are not running inference on a model when traversing one of those scenes, you are just rendering the splats.
At the very least it differs greatly from "world model" as understood in earlier robotics and AI research, wherein it referred to a model describing all the details of the world outside the system relevant to the problem at hand.
Very different, it would seem. Then again, it’s never been clear to me why LeCun believes that LLM architectures don’t inherently produce world models in the course of training.
IMO LLM more or less literally cannot do what they do without a world model, not least because much of what language is, is a protocol for making assertions about that model, testing the degree to which it is shared, and seeking to alter the model one carries of one's interlocutor's model.
To the "parrot people" I suggest, there is no more optimized mechanism for the inner layers of a network to approach than one which most parsimoniously models the world, so as to correctly emit tokens reflective of that.
dandersch|3 months ago
v9v|3 months ago
dr_dshiv|3 months ago
aaroninsf|3 months ago
IMO LLM more or less literally cannot do what they do without a world model, not least because much of what language is, is a protocol for making assertions about that model, testing the degree to which it is shared, and seeking to alter the model one carries of one's interlocutor's model.
To the "parrot people" I suggest, there is no more optimized mechanism for the inner layers of a network to approach than one which most parsimoniously models the world, so as to correctly emit tokens reflective of that.