Most teams build the thing, then try to measure it. That order is why so many systems get quietly switched off.
The labelled set is the spec
An eval suite written first is a specification. Before we ingest a document or write a prompt, we sit with practitioners and collect the real inputs: the top 200 questions a support team gets asked, the last 600 exceptions from an AP mailbox, 400 historical underwriting decisions with their outcomes. Labelling that set takes two days and it decides the whole design. In one AP engagement, categorising exceptions showed 82% fell into four mechanical buckets — which changed what we built entirely.
It also changes what “good” means. Without a labelled set, quality is whatever the last demo felt like. With one, you have a pass rate, and you can set the threshold yourself. That threshold belongs to the customer, not the vendor. We’ve had clients start at 98% on a decision workflow and 85% on a drafting workflow, and both were right, because the cost of being wrong differed by two orders of magnitude.
Three things beyond accuracy
Cost per run, with a ceiling. A system that works and costs $3.40 per item is a system finance will kill. Put the ceiling in CI.
Malformed and adversarial input. Missing fields, wrong currencies, duplicate submissions, a document in an unexpected language. Each becomes a test and a documented behaviour.
Drift. Scores move — model updates, document changes, seasonal input shifts. An alarm on a moving score is what separates a system you trust from one you hope about.
Why it is a commercial argument
The commercial argument matters more than the engineering one. A CXO cannot defend a system they can’t measure. Hand them a scoreboard — accuracy, cost, latency, per workflow, per month — and the conversation about expanding to the next workflow takes ten minutes.
Evals from day one. Not later.