A discovery by a giant is in some sense a new base vector in the space of discoveries. The interesting question is if a statistical machine can only perform a linear combination in the space of discoveries, or if a statistical machine can discover a new base vector in the space of discoveries.. whatever that is.
For sure we know modern LLMs and AIs are not constrained by anything particularly close to simple linear combinations, by virtue of their depth and non-linear activation functions.
But yes, it is not yet clear to what degree there can be (non-linear) extrapolation in the learned semantic spaces here.
nextaccountic|17 days ago
Kuxe|16 days ago
D-Machine|16 days ago
But yes, it is not yet clear to what degree there can be (non-linear) extrapolation in the learned semantic spaces here.
pram|17 days ago
TheCycoONE|17 days ago