LLM Evaluations: Automated Evals, Human Review

Shipping an LLM-powered feature without evaluations is shipping without tests.

LLM Evaluations: The Test Suite Every AI Product Needs Before It Ships

LLM evaluations (evals) are the systematic measurement of an AI system's output quality against a fixed set of inputs and expected characteristics. They are the equivalent of unit and integration tests for stochastic systems where the answer is not deterministic. Every meaningful change — new prompt, new model, new retrieval source, new context window — should run against the eval suite before it ships to users. Skipping evals is the modern equivalent of skipping tests: it works right until it doesn't.

The four types of eval

(1) Ground truth — you know the correct answer (extraction, classification, structured output). Score by exact match, F1, or JSON schema validation. Cheap, deterministic, use for anything closed-form. (2) Reference-based — you have a gold response written by a human; score similarity (BLEU, ROUGE, embedding cosine, LLM-as-judge). Useful for summarization and translation. (3) Reference-free — no ground truth; evaluate output properties directly (faithfulness to source, no hallucination, tone, safety) via rubric-based LLM-as-judge or classifier models. (4) Human evaluation — expert or crowdsourced ratings on a Likert scale. Slow, expensive, the ultimate ground truth for subjective quality.

Building the eval set

Start with 30-50 real production inputs that cover the most common patterns and the most painful edge cases. Categorize (query type, difficulty, expected behavior). Grow to 200-500 as the product matures. Include: (a) golden examples that must pass every time, (b) edge cases that recently broke, (c) adversarial prompts (jailbreaks, prompt injection, out-of-scope requests). Do NOT use production data unvetted — you will encode PII and biases into the eval set. Curate deliberately; document the rationale for each entry. An eval set is a strategic asset — treat it like source code, versioned in git.

LLM-as-judge, done carefully

Using a strong model (Claude, GPT) to grade another model's output is powerful and easy to get wrong. Best practices: (1) Rubric-based, not opinion-based — give the judge specific criteria with worked examples. (2) Chain-of-thought before the score — force the judge to reason before rating. (3) Pairwise comparison ('A or B, which is better?') is more reliable than absolute rating. (4) Verify judge calibration — sample 50 judge scores, have humans rate the same items, compute agreement. Below 80% agreement, refine the rubric. (5) Rotate judge models occasionally; don't over-fit to one judge's biases.

Running evals in CI/CD

Every prompt or model change opens a PR; CI runs the eval suite; results post to the PR as a diff vs. main. Regressions on golden examples block merge; regressions on statistical metrics require reviewer sign-off with justification. Tools: OpenAI Evals, Promptfoo, LangSmith, Braintrust, Humanloop, Weights & Biases Weave, or a homegrown harness (a few hundred lines of Python). Cost: evals against a 200-item set on a frontier model run $2-10 per pass; run on every PR touching prompts, plus nightly against production traffic samples.

Production monitoring closes the loop

Evals catch pre-deploy regressions; production monitoring catches everything else. Log every LLM call with input, output, model, prompt version, latency, cost, and (where possible) a downstream signal (did the user accept the suggestion, edit it heavily, or reject). Sample a fraction of production outputs into a review queue where humans grade them weekly. Feed graded examples back into the eval set. Alert on: rising latency (model degradation), rising rejection rate (quality regression), rising cost per session (context bloat), hallucination-detector spikes. The loop — production sample → human grade → eval set — is the single most important habit for teams shipping LLM features.

Frequently asked questions

How large should the eval set be?
Start with 30 items to get the pipeline working. Reach 150-300 for meaningful statistical resolution on regressions. Beyond 500, gains diminish per item — spend the effort on quality (diversity, edge cases, human-verified labels) rather than sheer count.
Can we evaluate a RAG system with these methods?
Yes, with additions: separately evaluate retrieval (did the right documents come back? use precision/recall against ground truth) and generation (given the retrieved context, is the answer faithful? use LLM-as-judge with a rubric that penalizes hallucinations and unsupported claims). Frameworks like RAGAS provide these specifically.
How do we handle non-deterministic outputs?
Run each eval item N times (N=3-5), aggregate. For classification-style tasks, majority vote or averaged confidence. For open-ended, aggregate rubric scores. Set temperature=0 for the deterministic evals; keep the production temperature for the tests that measure user-visible behavior.

Related fundraising guides (40)

Investor directory · Fundraising library · Articles A–Z · Company funding database