429 Too Many Requests from an AI API: retries with backoff

A 429 Too Many Requests from an AI API means the request was rejected because you sent too much too fast, or hit a limit on your account. The fix is to wait and retry with exponential backoff and jitter, to use the Retry-After header when the response has one, and to cap how many requests your agent sends in parallel. Do not retry a 429 that is really a spend or balance limit: waiting will not fix it.

This guide is based on the official rate-limit guides of OpenAI and Anthropic, both checked on 22 October 2026. Neither page shows a last-updated date.

What a 429 means#

OpenAI describes a 429 as a request rejected for exceeding a rate limit (rate_limit_error, with the code slow_down when traffic ramps up too fast), or because a hard spend limit was reached. Anthropic uses rate_limit_error for 429 as well. A spend-cap 429 has no retry-after header and keeps failing until access resumes.

Two neighbours are easy to confuse with it. OpenAI's 503 server_is_overloaded and Anthropic's 529 overloaded_error are overload conditions, not your rate limit. They are usually handled with the same retry loop, but they say something different about the cause.

So there are three cases, and each needs a different reaction:

If your balance ran out, read the "not enough tokens" error instead. For network-level failures such as timeouts and refused connections, see common connection errors.

Read the headers before you guess a delay#

OpenAI documents these headers on rate-limited responses: Retry-After, and x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests with -tokens variants. Other APIs may send a subset or none of them, so write the client to use Retry-After when it is present and fall back to your own schedule when it is not.

OpenAI's guidance on retries, in short:

• Treat Retry-After as the minimum wait and add a small random delay on top.
• Without it, use exponential backoff with jitter.
• Cap both the number of attempts and the total retry time.
• Do not stack your own retries on top of the SDK's retries.
• Do not retry quota or billing errors.
• Failed requests still count toward the per-minute limit, so an aggressive loop makes things worse.
• For streaming, do not replay a request after you have already consumed output from it.

Python: backoff with jitter#

The official openai package works against any OpenAI-compatible endpoint. Turn off the SDK's own retries (max_retries=0) so they do not multiply with yours. The id of the model comes from the CheapTokens docs; ids differ between providers.

import os
import random
import time

from openai import OpenAI, APIStatusError

client = OpenAI(
    base_url="https://api.cheaptokens.dev/v1",
    api_key=os.environ["CHEAPTOKENS_API_KEY"],
    max_retries=0,  # we retry ourselves
)

RETRYABLE = {429, 500, 502, 503, 504}


def chat_with_retry(model, messages, max_attempts=5, max_total=60.0):
    started = time.monotonic()
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except APIStatusError as err:
            last = attempt == max_attempts - 1
            if err.status_code not in RETRYABLE or last:
                raise
            header = err.response.headers.get("retry-after")
            if header and header.replace(".", "", 1).isdigit():
                delay = float(header) + random.uniform(0, 1)  # minimum wait + jitter
            else:
                delay = random.uniform(0, min(30.0, 2.0 ** attempt))  # full jitter
            if time.monotonic() - started + delay > max_total:
                raise
            time.sleep(delay)

The upper bound of the delay without a header grows 1, 2, 4, 8 seconds (capped at 30), and random.uniform picks a random value below it, so many workers do not all wake up at the same moment. Retry-After may also be an HTTP date; the code above handles the numeric form and otherwise falls back to backoff.

JavaScript: the same loop#

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.cheaptokens.dev/v1",
  apiKey: process.env.CHEAPTOKENS_API_KEY,
  maxRetries: 0, // we retry ourselves
});

const RETRYABLE = new Set([429, 500, 502, 503, 504]);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

export async function chatWithRetry(model, messages, maxAttempts = 5, maxTotalMs = 60000) {
  const started = Date.now();
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      return await client.chat.completions.create({ model, messages });
    } catch (err) {
      const last = attempt === maxAttempts - 1;
      if (!RETRYABLE.has(err.status) || last) throw err;
      const header = Number(err.headers?.get?.("retry-after"));
      const delayMs = Number.isFinite(header) && header > 0
        ? header * 1000 + Math.random() * 1000 // minimum wait + jitter
        : Math.random() * Math.min(30000, 2 ** attempt * 1000); // full jitter
      if (Date.now() - started + delayMs > maxTotalMs) throw err;
      await sleep(delayMs);
    }
  }
}

When it works, a rate-limited call simply takes longer and then returns a normal completion. When it still fails after the last attempt, the original error is raised, so your code can log it or queue the job for later.

How agents should handle 429#

An agent loop sends many short calls, and several agents or tool calls can run at once. Retrying alone is not enough; bring the request rate down at the source:

1. Cap concurrency. A semaphore of 3–5 parallel calls is a sensible start; raise it only while errors stay rare.
2. Share one limiter. If several workers use the same key, they share the same limits, so one retry loop per worker is not enough: use one queue or one semaphore for the process.
3. Back off globally. When one call gets a 429 with a delay, pause the whole pool for that time rather than only the failing call.
4. Keep requests smaller. Fewer tokens per request means less pressure on token-per-minute limits: trim history and tool output. See how to estimate the cost of an AI agent for where the tokens go.
5. Stop on non-retryable errors. A 401, 400, or a balance error will not fix itself; fail fast and report.

import asyncio

limit = asyncio.Semaphore(4)  # at most 4 requests in flight


async def run(job):
    async with limit:
        return await call_model(job)  # your async call with the retry loop above

Run agent loops on CheapTokens#

For agent loops with many short calls, CheapTokens offers MiniMax-M2.7: one price at any time of day, a 180K context, up to 8,192 tokens per answer, and one balance for all models. Output costs $0.15 per 1M tokens, and in total, with input and output counted together, it is cheaper than OpenRouter. It works through the OpenAI-compatible endpoint https://api.cheaptokens.dev/v1. The code above already points at this endpoint; set the model id from the CheapTokens docs and run it.

Live prices for a 500,000-input, 100,000-output workload:

See all prices, the setup docs, and create a key to send your first request.

If it still fails#

• 429 on the very first request. Read the error type and message in the response body: a spend or account-limit 429 keeps failing on retry until the limit is raised or access resumes, while a rate-limit 429 clears after a pause.
• 429 keeps coming with a long loop. You may be stacking retries: SDK retries times your retries. Set the SDK to zero, as in the examples.
• Errors come in bursts at the start of a batch. Ramp up gradually instead of firing all workers at once; OpenAI reports slow_down for traffic that grows too fast.
• Streaming call failed midway. Do not replay it blindly after you consumed output; decide whether to restart the whole answer or continue from what you have.
• No Retry-After in the response. The code falls back to backoff with jitter, so no change is needed.