[mannewalis]

But OpenAI's model isn't open source, how would they distill knowledge without direct access to the model?

[layer8]

You don’t need direct access for LLM distillation, just regular API access.

[mannewalis]

ok I looked it up and have a better understanding now.