Guide
Last updated: 26 September 2026
An agent is not production-ready because a demo worked. Readiness is a checklist you can fail. This is the one Zenovae uses before release.
An agent is production-ready when success is defined, traces exist, tool failures are visible, and a human owns the exception path. A working demo is not one of the criteria.
A demo shows one path through the task, run by the person who built it, on the inputs they chose. Production is every path, run by strangers, on inputs nobody previewed. The gap between those two is where agents fail quietly: wrong tool called, partial completion reported as success, cost spiking on retries.
The task, the success condition, and the failures that block release are written down before launch. 'It uses a stronger model' is not a success condition. If nobody can write the condition, nobody can score the agent.
Every run records what the agent saw, which tools it called, what they returned, and what it did next. No traces means every production question becomes an argument about memory.
A failed tool call, a timeout, and an empty result are distinct events, each surfaced somewhere a human reads. An agent that swallows a tool error and answers anyway is worse than no agent.
There is a named person, a handoff rule, and a channel. When the agent is unsure, something reaches that person. 'It escalates to the team' is not an owner.
A set of scored tasks stands between the agent and production. The gate is pass/fail, and the failures that block release were defined in week one, not discovered in week ten.
Not token spend — spend divided by completed tasks. An agent that is up and wrong still burns tokens. This number tells you whether it is up and right, affordably.
That is the normal state of a pre-launch agent, and it is fixable in a bounded window. The Agent Reliability Audit runs exactly this checklist against a live or pre-launch system over 10 business days and returns a written score plus the three changes that would move it.
Check six things: success is defined in writing, traces exist, tool failures are visible, a human owns the exception path, release is gated on an eval set, and cost per successful task is measured. Failing any one means it is not ready, no matter how good the demo looked.
No. A demo is one path, on chosen inputs, run by the builder. Production is every path, on real inputs, run by strangers. The criteria above exist because demos systematically hide the failure modes that production exposes.
Missing traces. Without them, task success cannot be scored, tool failures cannot be seen, and cost per successful task cannot be computed. Every other check depends on having the run history.
A named person with a defined handoff rule and a channel, not 'the team'. If the agent cannot complete the task or is unsure, something must reach that person with enough context to act.
The Agent Reliability Audit runs these criteria against your live or pre-launch agent over 10 business days and returns a written score.
Start a scoping call