Evaluating LLMs
Why is evaluating LLM output hard?
Generation is open-ended: there's rarely one correct answer, many phrasings are equally good, and quality spans correctness, helpfulness, style, and safety. So a single accuracy number doesn't capture it.
- “Why not just exact match?” — Two correct answers can share no words; exact match punishes valid paraphrases.
What metrics and benchmarks are used?
Match the method to whether the task has a reference answer.
For QA, classification, code (pass@k on unit tests).
Standardized suites for knowledge, reasoning, coding.
Side-by-side human preference (e.g. Elo from pairwise battles).
What is LLM-as-a-judge, and its risks?
Using a strong LLM to score or rank another model's outputs against a rubric. It's scalable and correlates reasonably with human judgment — but it has biases.
Judges favor longer answers, the first option presented (position bias), and their own style (self-preference). Mitigate with randomized order, clear rubrics, and human spot-checks.
How do you evaluate a RAG system?
Evaluate retrieval and generation separately, since either can fail.
- Recall@k, precision@k, MRR / nDCG — did it fetch the right chunks?
- Faithfulness (grounded in sources, no hallucination) and answer relevance.
What is benchmark contamination?
If a benchmark's test data leaked into the model's training corpus, high scores reflect memorization, not ability. It's the LLM-scale version of a train/test leak.
- “How do you guard against it?” — Use held-out / freshly created test sets, private eval suites, and report results on tasks released after the model's training cutoff.