[whakim]

Ok, but what was the cost of labor put into curation of the training dataset and performing the fine-tuning? Hasn’t the paper’s conclusion been repeatedly demonstrated - that it is possible to get really good task-specific performance out of fine-tuned smaller models? There just remains the massive caveat that closed-source models are pretty cheap and so the ROI isn’t there in a lot of cases.

[selim-now]

If the cost of getting the model is $200, then the cost of the trade-off seems to be quite clear.

You are right that the labor is a factor, unless you use a platform like https://www.distillabs.ai/ then the process is automated. (I'm affiliated)