top | item 46982851

(no title)

thomasmg | 18 days ago

LLMs are good at predicting the next token. Basically you use them to predict what are the probabilities of the next tokens to be a, b, or c. And then use arithmetic coding to store which one matched. So the LLM is used during compression and decompression.

discuss

order

No comments yet.