[jlopes2]

I just did a talk with Jerry from LlamaIndex earlier this week. https://www.youtube.com/watch?v=eLXivBehPGo

Included here is a bit of the old tried and true: NDCG/MRR/Precision @k - what you really want for measuring your information retrieval systems.

But we also talk through a bit of the "new", how to use Evals to generate the building blocks for those metrics above. You will want both hand labels and the automated Evals in the end to evaluate your system.