Every stalled AI programme we’re called into has the same shape. A pilot that worked. A steering committee that approved. And nothing in production nine months later.
The post-mortem almost always blames the model, or “change management.” It’s usually neither. It’s the gap between producing an output and finishing a piece of work.
Where the work actually ends
A demo ends when the model returns text. The work ends when the invoice is paid, the ticket is closed, the policy is bound. Between those two points sit the write-back to the system of record, the approval nobody had authority to design, the exception path for the 18% that arrive malformed, the audit trail your controller needs at month-end, and someone on call when a downstream API returns 500 for an hour.
None of that is glamorous and all of it is the project. Integration is the product.
Nobody set the number
The second reason is measurement. A pilot is judged by whether a room was impressed. Production is judged by whether it’s still trusted in month four — and if you can’t answer “is it working?” with a number, trust decays to zero on the first visible mistake. That’s why we write failure tests before the happy path: accuracy against a labelled set, cost per run against a ceiling, latency against a budget, and an alarm when any of them moves.
The wrong workflow got scoped
RFPs name the workflow that’s easiest to describe. The expensive one is the one people are slightly embarrassed about — the shared mailbox, the Friday spreadsheet, the queue with no owner. Look in the exception queue, not the process diagram.
If you want a cheap test of whether a pilot will survive: ask who owns the runbook, what the rollback is, and what number gets checked on Monday. If nobody can answer, the model quality is irrelevant.
Jane from AP manually fixes invoices that fail OCR on Fridays. Fix Friday.