Evaluation & verification
LLM-as-judge, rubrics, and adversarial checks — the loop that separates a tool from a demo.
The difference between a demo and a tool is a verification loop. A demo produces something plausible; a tool proves it did the right thing. Verification is where you buy back the reliability that a single generation can't give you.
Verify against the original goal
The cheapest verification is re-stating the requirement and checking the output against it — in the same run. For a bug fix, that means proving the fix against the original failing case, not a new one the agent invented. Reproduce, then fix, then re-run the repro (exactly what repro-first-debugging enforces).
Judge with a rubric, not a vibe
When you need to score quality — a review, a summary, an answer — give the judge a rubric: the explicit criteria and what each level looks like. "Is this good?" is unstable across runs; "does this meet criteria A, B, C?" is repeatable. This is the core of using an LLM as a judge, and of how we eval skills.
Verify adversarially
For high-stakes claims, don't ask "is this right?" — ask a fresh check to refute it, and default to skepticism when uncertain. An independent adversarial pass catches plausible-but-wrong results that a confirming pass rubber-stamps.
Uncertainty is a first-class output
A good verification step is allowed to say "not sure." An agent that surfaces its doubt is more useful than one that always claims done — and it's a behavior you can prompt for directly.