“Should the agent do this on its own?” gets argued as a matter of principle. It’s a threshold, and thresholds are cheap to change if you build for it.
Write the not-allowed list first
The first artifact we write for any agent is not a prompt or a tool list. It’s the list of things it is not allowed to do without a human. That list determines everything downstream: which credentials it gets, which writes are staged versus committed, what triggers a hold, what a rollback looks like.
Gates are best expressed as a number your business already understands. In accounts payable it was dollars: above $25,000, a human approves with the extraction and the source document side by side; below it, the system commits and logs. In underwriting it was confidence: below the line, route to a senior reviewer. Both started conservative and loosened once the measured pass rate justified it — and because the threshold is configuration, loosening it took an afternoon, not a re-architecture.
Two practices that make loosening safe
A shadow lane. Run on production data, produce real outputs, commit nothing, for one to two weeks. Then compare the system’s decisions against your team’s. That comparison is what gets a workflow approved. It’s a far easier conversation than a debate about model capability.
Reversibility. Every action logged with inputs, reasoning and the identity that authorised it. Idempotent writes. Replay from any state. If you can’t undo an action, it doesn’t get automated yet, whatever the pass rate says.
Design for confidently wrong
One more thing worth designing early: what happens when the system is confidently wrong. Not “if” — confidently wrong is the normal failure mode. Who notices, how fast, what’s the blast radius, how do you undo it? Write that down before launch and test it before handover.
Autonomy isn’t a stance. It’s a dial, with an owner and a number next to it.