top | item 43962508

(no title)

_QrE | 9 months ago

How can you call this 'Absolute Zero' if you need to start with a pretrained LLM? From what I understand, this just proposes that you can take an existing LLM, have it generate tasks and solve the tasks, and have it learn from that. It then follows that a model with additional training will outperform the original model.

I'm assuming that I'm misunderstanding something, because this doesn't seem very novel?

Edit: Seems like a variant of adversarial training?

discuss

order

make3|9 months ago

if you could improve the LLM without any further data, it would count as absolute zero. I'm highly skeptical however personally.