[SEGyges]

i hate "stochastic parrot" because it's not even really meaningful

I think it's true that models are statistical, inasmuch as P(A|B) where B is the prior sequence is what the loss is computing, and that's statistical. It's just computing that function in an absurdly complex way, which involves creating topological representations of relationships, etc.

I agree that "just" autocomplete implies the wrong thing. It turns out autocomplete is amazing if you scale it up.

I think it's true that they reason and area creative but these are really hard points because people mean subtly different things when saying "reason" and "creative".

[Nevermark]

> I think it's true that models are statistical, inasmuch as P(A|B) where B is the prior sequence is what the loss is computing

That is true of any system that is designed to do anything, with imperfect accuracy. Looking at systems as black boxes from the outside, probabilities means something.

But from a clear box perspective, the internals of these models are not statistical systems, they are spacial/topological mapping systems. Nothing resembling P(A|B) or any similar statistical measure appears in any calculation, except when judging the system from the outside.

There are statistical systems, built from statistical methods. But that's another class of algorithms.