AI Agent Partial Failures: Retry, Compensate, or Escalate?
When an AI agent changes several systems and a later step fails, choose a safe forward retry, business compensation, or human handoff.
Direct answer: retry forward only when the failure is transient and the same step is safe to repeat. Compensate completed steps only when the business action has a defined, authorized inverse. If the result is ambiguous or the next action is irreversible, pause and have an operator reconcile it. Persist each step and its outcome before choosing a recovery path; do not let a model infer that a write failed just because a request timed out.
Consider a SaaS support workflow that corrects an account entitlement after an agent classifies a ticket and a support lead approves the change. The workflow updates the entitlement, records the correction in the CRM, and sends a customer response. If the CRM update fails after the entitlement has changed, should the workflow undo the entitlement or finish updating the CRM? That depends on whether the entitlement change was correct and whether the CRM step can safely resume. It is a business recovery decision, not a model retry.
Distributed-workflow guidance calls this kind of sequence a saga: local transactions run across services, and a later failure can lead to forward recovery or compensating actions. A saga does not make separate service writes atomic. It records progress and coordinates the path toward a consistent business outcome. (AWS Prescriptive Guidance: Saga patterns, Microsoft Learn: Saga design pattern)
Choose the recovery path from the known outcome
Treat each error according to what the workflow can establish about the target system and the business intent. These are design recommendations for the support-account example, not universal retry rules.
| What is known after the failure | Recovery choice | Example |
|---|---|---|
| The step did not take effect, and the error is transient | Retry that step with a bounded policy | The CRM service reports a temporary service error before applying the update |
| The step took effect, and the approved goal remains correct | Continue forward from the next incomplete step | The entitlement correction is confirmed, but the CRM record still needs its matching note |
| A completed change is wrong, and its inverse is safe and authorized | Run the defined compensation, then verify the result | Restore the previous entitlement only if no later change has superseded it |
| The write result is unknown, the action cannot be safely repeated, or compensation might overwrite newer work | Stop automated writes and route a reconciliation task to a person | The entitlement API timed out after dispatch and does not confirm whether access changed |
| A customer notification was already sent | Do not pretend it can be undone; correct the record and send an approved follow-up if needed | A follow-up corrects the mistaken message in the support history |
AWS distinguishes infrastructure failures, which may be handled by retrying forward, from application failures such as invalid business data, which may call for compensation. Microsoft’s saga guidance identifies a pivot or point of no return: after an irreversible step, the workflow may need to finish forward rather than try to reverse the whole sequence. (AWS Prescriptive Guidance: Saga patterns, Microsoft Learn: Saga design pattern)
The practical rule is to retry only a classified transient failure, compensate only a known completed action with a designed inverse, and pause when the state or authority is unclear. A “rollback” is usually another business operation. It can fail, conflict with later work, or be inappropriate once another system or person has acted.
Keep the model out of the recovery control loop
The model can classify a support request, draft a proposed sequence, or summarize an exception. The application should own authorization, step ordering, durable state, retry limits, and recovery transitions. Before a write, check that the proposed action is still permitted for this case and this actor. After a write, record the target, operation identifier, result, and any version or reference needed to reconcile it.
Use explicit states such as proposed, approved, started, confirmed, outcome-unknown, compensation-pending, compensated, manual-review, and complete. These are application-level states, not a vendor standard. They make it possible for a worker restart or operator handoff to distinguish “the call failed before dispatch” from “the target may have changed but the response was lost.”
For every step, write down:
- The system of record and exact business effect.
- The evidence required before the step is authorized.
- Which errors are retryable, and the limit and delay for those retries.
- How to confirm the result after an ambiguous timeout.
- Whether an inverse exists, who may approve it, and how its success is confirmed.
- The owner and next action if retry or compensation fails.
Azure’s compensating-transaction guidance recommends recording forward progress and the corresponding compensation information. Its example retries transient faults first, then considers compensation when forward progress is impossible; it also describes pausing for human review when the case is ambiguous or high impact. Compensation logic is application-specific, so the workflow must define what “undo” means for its own records and customer commitments. (Microsoft Learn: Compensating transaction pattern)
This control boundary complements other reliability decisions. The tool-retry guide explains how to handle one ambiguous write without duplicating it. The background-job guide covers when the caller should receive a job ID and status resource. The Dataverse outbox article covers reliably delivering an event after one Dataverse change commits. This article addresses the next question: once several systems have participated, which business recovery path should run?
Native workflows first; custom orchestration only for a gap
For a short, fixed workflow with supported connectors and a clear manual-review branch, a native low-code flow may be sufficient. Power Automate documents scopes and run-after conditions for try/catch-style error handling, plus per-action retry policies. Use those controls when the required sequence, error branches, and operator visibility fit the workflow. They do not decide whether reversing a particular entitlement or customer communication is correct; that rule still needs an explicit owner and a tested definition. (Microsoft Learn: Employ robust error handling in Power Automate, Microsoft Learn: Use scopes to organize cloud flows)
If the workflow already runs in AWS Step Functions, its native Retry and Catch controls can retry selected failures and route exhausted or unmatched failures to a recovery state. Azure Durable Functions provides code-defined long-running orchestrations that coordinate activities and persist progress. Evaluate those built-in paths before adding a separate coordinator. (AWS Step Functions: Handling errors, Microsoft Learn: Durable orchestrations)
Custom orchestration is justified only when a specific gap blocks the release contract—for example, the native flow cannot preserve the per-step evidence needed for reconciliation, express the required conditional compensation, or expose a safe human-review route. Document the gap and test it. If the existing tool can express the steps, approvals, retries, and recovery outcomes your workflow needs, use it rather than rebuilding the same control flow.
Release test matrix for a partially completed support workflow
Inject failures around each external side effect. Verify the durable record and the next authorized action, not only whether the workflow ends in a “failed” status.
| Test case | Expected result |
|---|---|
| Validation fails before the first write | No external system changes; the case reports the missing prerequisite |
| A confirmed update is followed by a transient failure | The workflow resumes at the incomplete step under its bounded retry rule |
| A write times out after dispatch | The workflow marks the outcome unknown and reconciles the target before another write |
| A later business rule rejects the requested correction | Only previously completed, explicitly compensable steps are reversed; the case records each result |
| A compensation action fails | The workflow retains compensation-pending state, applies its own safe retry policy, and alerts the named owner |
| Another operator changes the entitlement before compensation | The workflow detects the conflict and stops instead of overwriting the newer change |
| The workflow fails after customer notification | The original message remains in history; any follow-up is a separately approved action |
| A worker stops between target success and local confirmation | Recovery queries the target or reuses a target-supported operation identifier before proceeding |
Test the exception path as thoroughly as the success path. For conditional writes and concurrent changes, use the stale-write release checks. For broader evaluation criteria, see the production-ready agent checklist and the agent evaluation guide.
Where Zenovae helps
If an agent already changes records across several systems, map each authorized effect, failure outcome, compensating action, and operator handoff before expanding its write access. The Agent Reliability Audit can assess an existing or pre-launch workflow against that release contract. A new workflow that needs implementation belongs with the production agent build team.
Sources
- AWS Prescriptive Guidance: Saga patterns
- Microsoft Learn: Saga design pattern
- Microsoft Learn: Compensating transaction pattern
- Microsoft Learn: Employ robust error handling in Power Automate
- Microsoft Learn: Use scopes to organize cloud flows
- AWS Step Functions: Handling errors
- Microsoft Learn: Durable orchestrations
Need Help with Your AI Project?
At Zenovae, we build production-ready AI systems that scale. From OpenClaw setup to custom integrations, Mission Control workflows, and full-stack delivery, we can help you ship faster and avoid costly mistakes.
Let's Talk