Back to Blog
    AI Agent Reliability
    October 20, 20269 min

    AI Agent Partial Failures: Retry, Compensate, or Escalate?

    When an AI agent changes several systems and a later step fails, choose a safe forward retry, business compensation, or human handoff.

    AI Agent ReliabilityDistributed WorkflowsCompensating TransactionsProduction AISaaS Support

    Direct answer: retry forward only when the failure is transient and the same step is safe to repeat. Compensate completed steps only when the business action has a defined, authorized inverse. If the result is ambiguous or the next action is irreversible, pause and have an operator reconcile it. Persist each step and its outcome before choosing a recovery path; do not let a model infer that a write failed just because a request timed out.

    Consider a SaaS support workflow that corrects an account entitlement after an agent classifies a ticket and a support lead approves the change. The workflow updates the entitlement, records the correction in the CRM, and sends a customer response. If the CRM update fails after the entitlement has changed, should the workflow undo the entitlement or finish updating the CRM? That depends on whether the entitlement change was correct and whether the CRM step can safely resume. It is a business recovery decision, not a model retry.

    Distributed-workflow guidance calls this kind of sequence a saga: local transactions run across services, and a later failure can lead to forward recovery or compensating actions. A saga does not make separate service writes atomic. It records progress and coordinates the path toward a consistent business outcome. (AWS Prescriptive Guidance: Saga patterns, Microsoft Learn: Saga design pattern)

    Choose the recovery path from the known outcome

    Treat each error according to what the workflow can establish about the target system and the business intent. These are design recommendations for the support-account example, not universal retry rules.

    What is known after the failureRecovery choiceExample
    The step did not take effect, and the error is transientRetry that step with a bounded policyThe CRM service reports a temporary service error before applying the update
    The step took effect, and the approved goal remains correctContinue forward from the next incomplete stepThe entitlement correction is confirmed, but the CRM record still needs its matching note
    A completed change is wrong, and its inverse is safe and authorizedRun the defined compensation, then verify the resultRestore the previous entitlement only if no later change has superseded it
    The write result is unknown, the action cannot be safely repeated, or compensation might overwrite newer workStop automated writes and route a reconciliation task to a personThe entitlement API timed out after dispatch and does not confirm whether access changed
    A customer notification was already sentDo not pretend it can be undone; correct the record and send an approved follow-up if neededA follow-up corrects the mistaken message in the support history

    AWS distinguishes infrastructure failures, which may be handled by retrying forward, from application failures such as invalid business data, which may call for compensation. Microsoft’s saga guidance identifies a pivot or point of no return: after an irreversible step, the workflow may need to finish forward rather than try to reverse the whole sequence. (AWS Prescriptive Guidance: Saga patterns, Microsoft Learn: Saga design pattern)

    The practical rule is to retry only a classified transient failure, compensate only a known completed action with a designed inverse, and pause when the state or authority is unclear. A “rollback” is usually another business operation. It can fail, conflict with later work, or be inappropriate once another system or person has acted.

    Keep the model out of the recovery control loop

    The model can classify a support request, draft a proposed sequence, or summarize an exception. The application should own authorization, step ordering, durable state, retry limits, and recovery transitions. Before a write, check that the proposed action is still permitted for this case and this actor. After a write, record the target, operation identifier, result, and any version or reference needed to reconcile it.

    Use explicit states such as proposed, approved, started, confirmed, outcome-unknown, compensation-pending, compensated, manual-review, and complete. These are application-level states, not a vendor standard. They make it possible for a worker restart or operator handoff to distinguish “the call failed before dispatch” from “the target may have changed but the response was lost.”

    For every step, write down:

    • The system of record and exact business effect.
    • The evidence required before the step is authorized.
    • Which errors are retryable, and the limit and delay for those retries.
    • How to confirm the result after an ambiguous timeout.
    • Whether an inverse exists, who may approve it, and how its success is confirmed.
    • The owner and next action if retry or compensation fails.

    Azure’s compensating-transaction guidance recommends recording forward progress and the corresponding compensation information. Its example retries transient faults first, then considers compensation when forward progress is impossible; it also describes pausing for human review when the case is ambiguous or high impact. Compensation logic is application-specific, so the workflow must define what “undo” means for its own records and customer commitments. (Microsoft Learn: Compensating transaction pattern)

    This control boundary complements other reliability decisions. The tool-retry guide explains how to handle one ambiguous write without duplicating it. The background-job guide covers when the caller should receive a job ID and status resource. The Dataverse outbox article covers reliably delivering an event after one Dataverse change commits. This article addresses the next question: once several systems have participated, which business recovery path should run?

    Native workflows first; custom orchestration only for a gap

    For a short, fixed workflow with supported connectors and a clear manual-review branch, a native low-code flow may be sufficient. Power Automate documents scopes and run-after conditions for try/catch-style error handling, plus per-action retry policies. Use those controls when the required sequence, error branches, and operator visibility fit the workflow. They do not decide whether reversing a particular entitlement or customer communication is correct; that rule still needs an explicit owner and a tested definition. (Microsoft Learn: Employ robust error handling in Power Automate, Microsoft Learn: Use scopes to organize cloud flows)

    If the workflow already runs in AWS Step Functions, its native Retry and Catch controls can retry selected failures and route exhausted or unmatched failures to a recovery state. Azure Durable Functions provides code-defined long-running orchestrations that coordinate activities and persist progress. Evaluate those built-in paths before adding a separate coordinator. (AWS Step Functions: Handling errors, Microsoft Learn: Durable orchestrations)

    Custom orchestration is justified only when a specific gap blocks the release contract—for example, the native flow cannot preserve the per-step evidence needed for reconciliation, express the required conditional compensation, or expose a safe human-review route. Document the gap and test it. If the existing tool can express the steps, approvals, retries, and recovery outcomes your workflow needs, use it rather than rebuilding the same control flow.

    Release test matrix for a partially completed support workflow

    Inject failures around each external side effect. Verify the durable record and the next authorized action, not only whether the workflow ends in a “failed” status.

    Test caseExpected result
    Validation fails before the first writeNo external system changes; the case reports the missing prerequisite
    A confirmed update is followed by a transient failureThe workflow resumes at the incomplete step under its bounded retry rule
    A write times out after dispatchThe workflow marks the outcome unknown and reconciles the target before another write
    A later business rule rejects the requested correctionOnly previously completed, explicitly compensable steps are reversed; the case records each result
    A compensation action failsThe workflow retains compensation-pending state, applies its own safe retry policy, and alerts the named owner
    Another operator changes the entitlement before compensationThe workflow detects the conflict and stops instead of overwriting the newer change
    The workflow fails after customer notificationThe original message remains in history; any follow-up is a separately approved action
    A worker stops between target success and local confirmationRecovery queries the target or reuses a target-supported operation identifier before proceeding

    Test the exception path as thoroughly as the success path. For conditional writes and concurrent changes, use the stale-write release checks. For broader evaluation criteria, see the production-ready agent checklist and the agent evaluation guide.

    Where Zenovae helps

    If an agent already changes records across several systems, map each authorized effect, failure outcome, compensating action, and operator handoff before expanding its write access. The Agent Reliability Audit can assess an existing or pre-launch workflow against that release contract. A new workflow that needs implementation belongs with the production agent build team.

    Sources

    Need Help with Your AI Project?

    At Zenovae, we build production-ready AI systems that scale. From OpenClaw setup to custom integrations, Mission Control workflows, and full-stack delivery, we can help you ship faster and avoid costly mistakes.

    Let's Talk