Shipping an LLM-powered feature without evaluations is shipping without tests.
LLM evaluations (evals) are the systematic measurement of an AI system's output quality against a fixed set of inputs and expected characteristics. They are the equivalent of unit and integration tests for stochastic systems where the answer is not deterministic. Every meaningful change — new prompt, new model, new retrieval source, new context window — should run against the eval suite before it ships to users. Skipping evals is the modern equivalent of skipping tests: it works right until it doesn't.
(1) Ground truth — you know the correct answer (extraction, classification, structured output). Score by exact match, F1, or JSON schema validation. Cheap, deterministic, use for anything closed-form. (2) Reference-based — you have a gold response written by a human; score similarity (BLEU, ROUGE, embedding cosine, LLM-as-judge). Useful for summarization and translation. (3) Reference-free — no ground truth; evaluate output properties directly (faithfulness to source, no hallucination, tone, safety) via rubric-based LLM-as-judge or classifier models. (4) Human evaluation — expert or crowdsourced ratings on a Likert scale. Slow, expensive, the ultimate ground truth for subjective quality.
Start with 30-50 real production inputs that cover the most common patterns and the most painful edge cases. Categorize (query type, difficulty, expected behavior). Grow to 200-500 as the product matures. Include: (a) golden examples that must pass every time, (b) edge cases that recently broke, (c) adversarial prompts (jailbreaks, prompt injection, out-of-scope requests). Do NOT use production data unvetted — you will encode PII and biases into the eval set. Curate deliberately; document the rationale for each entry. An eval set is a strategic asset — treat it like source code, versioned in git.
Using a strong model (Claude, GPT) to grade another model's output is powerful and easy to get wrong. Best practices: (1) Rubric-based, not opinion-based — give the judge specific criteria with worked examples. (2) Chain-of-thought before the score — force the judge to reason before rating. (3) Pairwise comparison ('A or B, which is better?') is more reliable than absolute rating. (4) Verify judge calibration — sample 50 judge scores, have humans rate the same items, compute agreement. Below 80% agreement, refine the rubric. (5) Rotate judge models occasionally; don't over-fit to one judge's biases.
Every prompt or model change opens a PR; CI runs the eval suite; results post to the PR as a diff vs. main. Regressions on golden examples block merge; regressions on statistical metrics require reviewer sign-off with justification. Tools: OpenAI Evals, Promptfoo, LangSmith, Braintrust, Humanloop, Weights & Biases Weave, or a homegrown harness (a few hundred lines of Python). Cost: evals against a 200-item set on a frontier model run $2-10 per pass; run on every PR touching prompts, plus nightly against production traffic samples.
Evals catch pre-deploy regressions; production monitoring catches everything else. Log every LLM call with input, output, model, prompt version, latency, cost, and (where possible) a downstream signal (did the user accept the suggestion, edit it heavily, or reject). Sample a fraction of production outputs into a review queue where humans grade them weekly. Feed graded examples back into the eval set. Alert on: rising latency (model degradation), rising rejection rate (quality regression), rising cost per session (context bloat), hallucination-detector spikes. The loop — production sample → human grade → eval set — is the single most important habit for teams shipping LLM features.
Investor directory · Fundraising library · Articles A–Z · Company funding database