Agent Retry Policy in Production
Watch an agent fail at two in the morning and you will rarely see one retry. You will see three. The model SDK retries the request on its own, because that is its default. The tool wrapper catches the same error and tries again. Then the orchestrator marks the step failed and restarts it, replaying both retries underneath. Each layer behaved sensibly on its own. Together they turned one failed call into eight attempts, and the service at the other end — the one that was already struggling — got to meet every copy.
Our position is simple: a run gets one retry policy, owned by one layer, and every failure passes through it. Retries decided independently at each layer are not resilience. They are uncoordinated load, generated at exactly the moment the system can least afford it.
Retries multiply. They don't add
This arithmetic is old, and it still surprises people. The Amazon Builders' Library works it through for a five-deep stack of service calls with three retries at each layer: the load on the database at the bottom can increase 243 times. Nothing about the original failure changed. Each layer just added its own attempts, and attempts at stacked layers multiply instead of adding.
Google's SRE book tells the same story from the server's side, in its chapter on cascading failures. A backend that handles 10,000 requests per second receives 10,100. The extra hundred are rejected and retried, so the backend now sees 10,200, and the failing share grows by another hundred each round. Soon a smaller and smaller fraction of the traffic is original work. The rest is the recovery mechanism, feeding on itself.
Agents stack more layers than the services in those examples. A model call can be retried by the SDK, a tool call by its wrapper, a step by the runner, and a whole task by whatever scheduled it. A parent agent that respawns a failed child adds one more. Unless someone has deliberately chosen which of those layers owns the retry, the answer is all of them, and the multiplication above is the polite version of what happens.
Read the failure before you retry it
A status code is not a classification. Two responses can share a code and mean opposite things.
Take the 429. A rate-limit response from the Claude API can carry a retry-after header: the number of seconds to wait before retrying, and the documentation is explicit that earlier retries will fail. That response is an instruction to wait. But the same API returns a 429 when an organisation hits the spend cap on its usage tier, and that version never carries a retry-after header. Its error details carry a different code instead, and retrying it, SDK automatic retries included, fails until access resumes. Same status, same error type on the surface. One says wait — eighteen seconds, in the example below. The other says stop, because waiting does not raise a cap.
The other classes matter just as much. A malformed request or a permission error fails identically on the fourth attempt; retrying it spends money to learn nothing. An overloaded server is asking for less load, so a fast retry answers it with more. And a timeout is its own problem: the request may well have completed after the client gave up, so a call with side effects can do the work twice. If a call cannot be repeated safely, the honest options are to check what actually happened or to stop. Firing again and hoping is not a third option.
So the policy reads the failure first: what class is this, what did the server actually ask for, and is this call safe to send twice? Only then does it decide whether a retry exists at all.
One owner, one budget
With classification in place, the rest of the policy is three decisions.
One layer retries, and the others propagate the failure upward. Pick the layer closest to the call that failed (it has the error, the headers, and the context) and turn the rest off, including the quiet defaults inside SDKs and frameworks. A default you have not read is still a policy; it is just one nobody chose.
Retries get a budget of their own, separate from the per-request attempt cap. A cap of three attempts per call says nothing about what a thousand concurrent runs may retry in the same minute. The SRE book's example is a process-wide budget: 60 retries per minute in a process, and once the budget is spent, requests fail instead of retrying. The shape matters more than the number: retries across the whole system stay a small, bounded share of traffic, even while everything is failing at once.
And the wait itself is not ours to invent. When a server sends retry-after, that value is a minimum: wait at least that long, then add jitter (a random spread) so the hundred clients who were all told to wait the same number of seconds do not all return in the same one. OpenAI's rate-limit guidance adds a detail that closes the loop: unsuccessful requests count toward your per-minute limit, so a tight retry loop spends the very allowance it is waiting to recover.
For an agent run, all of this charges against the run's own budgets of steps, tokens, and time as well. A retry of a model call resends the context with it. A policy that ignores that cost will produce runs that are reliable, punctual, and far more expensive than the work they delivered.
What retries cannot fix
A retry policy hides one failure shape particularly well: the call that fails a third of the time and succeeds on the second attempt. Success metrics look fine. First-attempt success is the number that tells the truth, and it belongs on the same dashboard as the retry rate — a rising retry rate is usually a degrading dependency announcing itself early, if anyone is watching that dial.
There are honest limits on the other side, too. Stop retrying entirely, the circuit-breaker shape, and you trade one failure mode for another: a system that refuses to try can stay closed after the dependency has recovered, and Amazon's library warns that breakers add modal behaviour that is difficult to test and can slow recovery. And some work should never be retried late. If the caller needed an answer within thirty seconds, a retry scheduled at forty-five is spending money on work nobody will use. Deadlines have to travel with the request, so a layer deep in the stack can see that the run it is retrying for has already been given up on.
None of this argues for fewer retries in total. Transient failures are real, and a single well-placed retry absorbs most of them. It argues for retries that are classified, budgeted, and owned — placed the way you would place any other load on a system you are responsible for.
Work with us
If your agents retry at every layer by default, or a 429 in your logs could mean two different things and your code treats them as one, that is a contained piece of work: one policy, one owner, and failures your team can read at a glance. Tell us about the run that keeps coming back through Trida AI's intake, and we will help you give it a retry policy that helps instead of multiplying.