Agent Retry Logic: The Silent Cost Multiplier in Multi-Agent Systems

Most teams discover their agent retry problem the same way: a single runaway task, a Slack alert at 2am, and a token bill that's 10x the daily average. One misconfigured retry cascade across five agents can multiply your token spend by 8–12x in a single task execution — and the math isn't exotic, it's just compounding. The insidious part is that retries feel like reliability. You're being defensive. You're handling transient failures. The agent kept going instead of crashing. But in agentic systems, "kept going" and "quietly burned money" are often the same sentence.

Imagine a team building a travel booking agent: their orchestrator retries on every exception, their LLM client has its own default retry policy, and their tool layer adds one more. A single API timeout cascades into 27 LLM calls nobody planned for. This is the default behavior if you wire up a naive retry decorator around an orchestrator with no blast-radius controls.

Why Agent Retries Are Structurally Different From API Retries

In a standard microservice, a retry is cheap: you're resending a small JSON payload. Worst case, you hit a rate limit or get a duplicate write — both recoverable with idempotency keys.

In an agent, a retry is an LLM call. Often multiple LLM calls — one to decide what tool to use, one to call it, one to evaluate the result. If the agent uses a planning loop (ReAct, LATS, or any variant), a single "retry" might trigger a full re-plan that fans out to three sub-agents, each of which retries independently.

The compounding math is brutal:

• Agent A fails → retries 3 times → spawns Agent B and Agent C on attempt 2
• Agent B fails on a tool call → retries 3 times
• Agent C times out → retries 2 times with exponential backoff on each
• Total LLM calls from one logical "retry": 3 + (3×3) + (2×3) = 18 calls instead of 1

At $3–15 per million tokens for capable frontier models, and with agents consuming 5,000–30,000 tokens per call depending on context window usage, that cascade costs real money. The failure mode isn't a bug — it's the natural consequence of composing independent retry layers without a unified blast-radius policy.

The Three Failure Modes of Agent Retry Logic

1. Retry-on-everything

The classic mistake: wrap every LLM call in a tenacity-style retry with max_attempts=3 and call it resilience. The problem is that most agent failures aren't transient network errors — they're semantic failures. The model misunderstood the task, the tool returned unexpected schema, or the context window is saturated with irrelevant conversation history. Retrying the same call with the same context doesn't fix those. It just burns tokens three times.

The rule: only retry on retriable errors. Rate limits (429) and transient server errors (503) are retriable. Malformed output, tool schema mismatches, and context-length violations are not — they need a different recovery path (structured fallback, context compression, or escalation to a human).

python
# Distinguish retriable vs non-retriable failures
RETRIABLE_ERRORS = {"RateLimitError", "ServiceUnavailableError", "TimeoutError"}
NON_RETRIABLE_ERRORS = {"ContextLengthExceeded", "InvalidToolSchema", "MalformedJSON"}

def should_retry(error: Exception) -> bool:
    return type(error).__name__ in RETRIABLE_ERRORS

2. Inherited retry stacks

This happens when you compose agents from libraries that each bring their own retry logic. The orchestration framework retries. The LLM client retries. The tool execution layer retries. You end up with a 3×3×3 = 27x blast radius from a single failure, with no single layer aware the others exist.

The fix is to own retry at exactly one layer — the orchestrator — and strip or disable it everywhere below. If you're using LangGraph, LlamaIndex, or a custom orchestrator, audit every dependency for default retry behavior and set it explicitly to zero before implementing your own policy.

3. Infinite escalation without circuit breakers

The worst pattern: an agent that escalates to a more capable (and expensive) model on failure. Fail with a cheaper model, escalate to a mid-tier, fail again, escalate to the most expensive option — all within one task. Without a hard cap on escalation budget, a single edge-case input can drain your daily token budget before your monitoring fires.

This is not a theoretical concern. Escalation-based routing is a legitimate cost optimization strategy — I've written about the economics of routing in GPT-5.6 Luna at $0.20: Reprice Your Agent Routing Stack Now — but it must be bounded. Set a maximum escalation tier per task and enforce it at the orchestrator level before any agent call is made.

A Practical Retry Policy for Multi-Agent Systems

Here's the decision framework I use when designing retry logic for agent systems:

Error Type	Retry?	Max Attempts	Backoff	Recovery Path
Rate limit (429)	Yes	3	Exponential + jitter	Resume from checkpoint
Server error (503)	Yes	2	Fixed 2s	Resume from checkpoint
Timeout	Yes	2	Linear 5s	Resume from checkpoint
Context length exceeded	No	0	—	Compress context, re-plan
Malformed tool output	No	0	—	Fallback schema or human review
Model refuses (safety)	No	0	—	Escalate to human
Semantic failure (bad output)	No	0	—	Re-prompt with correction, count separately

A few non-obvious rules baked into this table:

• Checkpoint before every tool call. If you're retrying, resume from the last successful state, not from the top. This is the difference between "retry the tool call" and "retry the entire 10-step agent plan."
• Semantic retries are not retries — they're re-prompts. Track them separately in your telemetry. They have a different cost profile and a different failure signal.
• Always add jitter to exponential backoff. Without jitter, all retried agents in a fleet hit the provider simultaneously, converting a transient error into a sustained rate-limit storm.

Checkpoint State Is Not Optional

The reason retry logic in agents is hard isn't the retry policy itself — it's that most teams don't checkpoint agent state. Without checkpoints, a retry means starting the entire task from scratch — expensive and semantically dangerous, because the agent may have already made side effects: sent an email, created a booking, updated a record.

Consider what happens in a multi-step travel agent (a pattern directly relevant to the systems I work on at Etera AI): if step 6 of 10 fails and there's no checkpoint, a naive retry re-executes the flight search, the hotel lookup, the availability check, and the pricing call — all over again, each with its own token cost. With checkpoints, you resume from step 6. The difference compounds fast across a fleet of concurrent tasks.

Every step that has a side effect needs a checkpoint before and a confirmation after. The checkpoint is cheap — it's a write to Redis or Postgres. The alternative is idempotency guarantees across every tool you call, which is far harder to retrofit.

A minimal checkpoint looks like:

python
@dataclass
class AgentCheckpoint:
    task_id: str
    agent_id: str
    step_index: int
    state: dict          # serialized working memory
    completed_steps: list[str]
    side_effects: list[dict]  # log of external mutations
    created_at: datetime

def safe_execute_step(agent, step, checkpoint_store):
    checkpoint = AgentCheckpoint(
        task_id=agent.task_id,
        agent_id=agent.id,
        step_index=step.index,
        state=agent.state.snapshot(),
        completed_steps=agent.completed_steps,
        side_effects=agent.side_effects_log,
        created_at=datetime.utcnow()
    )
    checkpoint_store.write(checkpoint)
    result = step.execute()
    checkpoint_store.mark_complete(checkpoint.task_id, step.index)
    return result

If you can't replay an agent's execution from any checkpoint, your debugging story is guesswork. For a deeper treatment of the full observability stack that makes checkpoints actionable, see Agent Observability: The Telemetry Gap Killing Your Agentic Stack.

Token Budget Enforcement Is a Hard Constraint, Not a Soft Goal

Every agent task should have a token budget — a hard upper bound on total tokens consumed across all sub-agents, tools, and retries. When the budget is hit, the task fails gracefully with a budget-exceeded status, not with a retry.

Most frameworks don't enforce this natively. You have to wire it yourself:

python
class BudgetedOrchestrator:
    def __init__(self, task, token_budget: int):
        self.task = task
        self.token_budget = token_budget
        self.tokens_consumed = 0

    def record_usage(self, tokens: int):
        self.tokens_consumed += tokens
        if self.tokens_consumed > self.token_budget:
            raise BudgetExceededError(
                f"Task {self.task.id} exceeded budget: "
                f"{self.tokens_consumed}/{self.token_budget} tokens"
            )

Set your budget based on the p95 token consumption of successful task completions, then multiply by 1.5 as headroom. Track this per task type, not globally — a research agent and a form-filling agent have radically different consumption profiles.

Retries that consistently exceed budget are a signal, not just a cost. If a task type repeatedly hits the ceiling before completing, the problem is task design, context window management, or the agent loop — not the retry policy. Fix the root cause, don't raise the cap.

The One Rule That Contains Most of the Blast Radius

If you implement nothing else from this article, implement this: no agent may retry itself more than once without a checkpoint write in between.

That single constraint prevents the most common runaway pattern — an agent re-entering its own loop without persisting state — and forces you to think about what "retry" actually means at each step. It doesn't eliminate the problem, but it contains it. In production, containment is often more valuable than prevention.

What to Actually Do

1. Audit your current retry surface. Grep your codebase for every retry decorator, max_retries parameter, and backoff configuration across all layers — orchestrator, LLM client, tool runners. Overlapping retry stacks are nearly universal in agent systems assembled quickly from multiple libraries.

2. Classify your errors before you set your policy. Split failure modes into retriable (transient infrastructure) vs non-retriable (semantic, schema, budget). Route non-retriable failures to a separate recovery path, not back into the retry loop.

3. Add checkpointing before any side-effecting tool call. Start with the highest-risk tools first — anything that mutates external state (APIs, databases, messaging). Redis with a 24-hour TTL is sufficient for most agentic tasks.

4. Enforce a token budget per task type. Instrument your current agents for two weeks to get p50/p95 baselines, then set hard caps at p95 × 1.5. Wire a BudgetExceededError that fails the task cleanly rather than silently burning budget.

5. Add retry-specific telemetry. Track retry count, retry reason, and tokens consumed on retry separately from normal execution. If your retry spend exceeds 10% of total agent token spend, you have a design problem somewhere upstream.

Retries are not resilience. Done wrong, they're a slow-motion outage with a very expensive invoice attached.