All insights
A demo agent needs a good prompt. A production agent needs retries, budgets, an audit trail and a human who owns the failure. Here is the operating checklist we hand every team before an agent touches a real customer workflow.
A demo agent has one job: look good for four minutes. A production agent has a different job: keep working at 3am, on the ugliest input a customer can produce, while someone is on the phone. The distance between those two is not a better prompt. It is an operating model.
This is the checklist we hand every team before an agent touches a real workflow. None of it is exotic. All of it gets skipped under deadline pressure.
Every step can fail, so every step needs a plan
Tool calls time out. APIs return half a payload. The model picks the wrong tool with complete confidence. Each of those needs a defined retry policy, a cap on how many times it can retry, and a defined outcome when the cap is hit. "It just tries again" is not a policy.
We write the failure path first. What does the customer see when step three of five fails? Who is told? What state is the record left in? If those questions have no answer, the agent is not ready.
Budgets, not hopes
An agent that loops is an agent that spends. Every run gets a token budget, a wall-clock budget and a tool-call budget, and it stops when any one is exhausted. The budget is set from the p95 of real runs, not from what feels generous.
The same numbers feed the dashboard. If the median run doubles overnight, something changed upstream, and you want to know before finance does.
An audit trail a human can read
Every run records what the agent saw, what it decided, which tools it called with which arguments, and what came back. Not for compliance theatre: for the Tuesday when a customer asks why their invoice was changed and someone has to answer in under an hour.
The trail is written in plain language alongside the raw JSON. Support should be able to read it without an engineer.
Someone owns the failure
The last item is organisational. An agent in production has a named owner who is paged when it misbehaves, who reviews the weekly sample of runs, and who has authority to switch it off. Without that person, every incident becomes a debate about whose problem it is.
Get these four right and the model choice matters far less than people expect. Skip them and no model will save you.
