top | item 44019808

(no title)

mondrian | 9 months ago

Also in the vicinity: https://www.anthropic.com/research/tracing-thoughts-language...

There's also distillation, where you can drastically improve a small model by training it on chains of thoughts of larger models. You can't achieve the same performance by training on original human texts. This suggests that those chains of thoughts reliably contain "densely packed reasoning", meaning the LLM probably has developed internal clusters of "reasoning circuitry", loosely speaking.

discuss

order

No comments yet.