If LLMs were good at summarization, this wouldn't be necessary. Turns out a stochastic model of language is not a summary in the way humans think of summaries. Thus all this extra faff.
What are the good models for summarization? I have found all, particularly local models, to be poor. Is there a leaderboard for summarization somewhere?
How do you evaluate quality ? Also I suspect the performance between models would varry between datasets. Heck it would vary on same model/source if you included that your mother was being held hostage and will be killed unless you summarize the source correctly :).
I think you are still stuck with try if it works for you and hope it generalizes beyond your evaluation.
sroussey|11 months ago
rafaelmn|11 months ago
I think you are still stuck with try if it works for you and hope it generalizes beyond your evaluation.