top | item 47125800

(no title)

gcanyon | 6 days ago

This speaks very much to the idea that LLMs are in some sense a ridiculously effective, somewhat lossy, compression algorithm that has been applied to the whole internet.

discuss

order

in-silico|6 days ago

It's a good way to frame base models that have only been pretrained.

However, modern frontier models have undergone rounds of fine-tuning, RLHF (reinforcement learning from human feedback), and RLVR (RL from verifiable rewards) that turn them into something else. The compressed internet is still in there, but it's wrapped in problem-solving and people-pleasing circuitry.

vizzier|6 days ago

I've thought of them for a while as just a really complicated indexing strategy.

r_lee|6 days ago

I mean, the transformer is basically like a big query engine and the model is the dataset + some logic or whatever

it's kind of like that by definition, with the whole Attention stuff etc.