Evals are the new unit tests: build the harness before you trust the model
Evaluation sets, graders, regression and what "better" means for an AI workflow
The question: How do you know a prompt change, a model swap or a retrieval tweak made the system better — rather than just different?
Every AI system is tuned by people who cannot tell, without a harness, whether their last change helped one case and broke ten. GDPval (2025) showed what it takes to judge models on real work; this lesson brings that discipline down to a single workflow.
What the lesson covers
An **eval set** is a fixed collection of real inputs with known-good outputs or grading criteria, run on every change to the prompt, the model, the retrieval or the tools. It is to an AI workflow what unit tests are to software: the thing that turns "I think it is better" into a number that moves or does not. Thirty realistic cases beat three hundred synthetic ones, because the set must look like the work.
Grading has three shapes. **Exact or programmatic checks** where the answer is determinable (a number, a category, a schema). **Rubric grading by a second model** — "LLM as judge" — for open outputs, with the rubric written by a human and spot-audited against human grades. And **human grading** on a sample, which is the ground truth the other two are calibrated against. GDPval's method — blind expert comparison of model deliverables against human ones, across 44 occupations — is the gold standard version of the third.
The metrics follow the pipeline. For retrieval: did the right passages come back (recall, precision). For grounding: is every claim supported by the sources (faithfulness). For the task: did it answer what was asked (relevance), and did it meet the rubric (quality). For the system: cost and latency per case, because a better answer that costs five times more is a different product.
The habit that makes it a harness rather than a demo is **regression**: every change runs the whole set, and a change that improves the case you were looking at while degrading the aggregate is rejected. The most common AI "improvement" is a prompt tweak that fixed one answer and silently broke ten, and only a harness can see it.
Build the harness before you trust the model, and keep it alive: add every production failure to the set as a new case, so the harness learns what the world taught you. A team with a living eval set can swap models in an afternoon with evidence; a team without one is guessing with confidence.
Key points
- An eval set is a fixed set of real cases with graders, run on every change — the unit tests of an AI workflow.
- Three grader shapes: programmatic checks, LLM-as-judge with a human-written rubric, and human grading as the calibration ground truth.
- Metrics follow the pipeline — recall/precision, faithfulness, relevance, quality — plus cost and latency per case.
- Regression is the habit: a change that helps one case and hurts the aggregate is rejected.
- Add every production failure to the set; a living harness lets you swap models with evidence.
Framework — Cases → Grader → Metrics → Regression → Add the failures
Thirty real cases, a grader calibrated on human judgement, the pipeline's metrics plus cost, every change run against the whole set, every production miss added back in.
The lab
Build a real eval harness for one workflow and use it to judge a change.
Open this lesson, its lab and its quiz