OpenAI Python SDK timeout and retries: set them for CheapTokens

Your script called the model, nothing came back, and ten minutes later it is still waiting. That is the OpenAI Python SDK's default: a 10-minute timeout and 2 retries. To change it, pass timeout=30.0 and max_retries=2 to OpenAI(...), override them per call with client.with_options(...), and cap the answer with max_tokens.

The SDK's README (checked 27 October 2026) documents all of this. Below it is applied to the CheapTokens endpoint, https://api.cheaptokens.dev/v1, which speaks the OpenAI Chat Completions format. Details of that setup are in Your first API request with CheapTokens and the docs.

What the SDK does when you set nothing#

Two defaults matter. The timeout is 10 minutes. Retries are 2, with a short exponential backoff, for connection errors and for 408, 409, 429 and 5xx responses, and only when the request body can be safely resent. A request that times out is retried too.

So each attempt gets its own timeout, and the worst case is roughly the timeout multiplied by the number of attempts, plus the pauses between them:

• default: 600 × 3 = 1800 seconds, half an hour
• timeout=30.0, max_retries=2: 30 × 3 = 90 seconds

Say a web endpoint has to answer within 30 seconds. With the defaults it can hang for half an hour before your code sees an error. Try it on another setting before reading on.

Set timeout and max_retries on the client#

The simplest fix is to set both once, when you create the client:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.cheaptokens.dev/v1",
    api_key=os.environ["CHEAPTOKENS_API_KEY"],
    timeout=30.0,      # seconds per attempt
    max_retries=2,     # 0 turns retries off
)

resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash-0731",
    messages=[{"role": "user", "content": "Summarize this ticket in two sentences: ..."}],
    max_tokens=800,
)
print(resp.choices[0].message.content)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)

max_retries must be a non-negative integer. timeout takes a float, or an httpx Timeout object if you want separate limits for connecting and reading. The README of the current main branch imports httpx2 for that, older versions used httpx, so run pip show openai and check which one your install uses. A plain float needs no import and works either way.

Use the full model id from the CheapTokens docs, deepseek-ai/DeepSeek-V4-Flash-0731 here: ids differ between providers. This model takes up to 32,768 output tokens per answer and has a 380K context, which is why the max_tokens line above is there on purpose. More on it below, and on the model in cheap DeepSeek.

Override it for one request#

One client, different jobs. A chat reply should give up fast, a nightly summary can wait. Instead of creating a second client, derive one for the call:

quick = client.with_options(timeout=5.0, max_retries=0)
resp = quick.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash-0731",
    messages=[{"role": "user", "content": "Reply with one word: ok"}],
    max_tokens=5,
)

with_options returns a copy with the new values and leaves client alone. Setting max_retries=0 is the right call when your own code already retries, for example a job queue that re-enqueues the task. Otherwise the two layers multiply: 3 SDK attempts inside 3 queue attempts is 9 requests for one task.

Catch the errors that matter#

After the retries run out, the SDK raises an exception. Every error class inherits from openai.APIError, and failed requests carry request_id, which is what you log:

import openai

def ask(prompt: str) -> str | None:
    try:
        r = client.chat.completions.create(
            model="deepseek-ai/DeepSeek-V4-Flash-0731",
            messages=[{"role": "user", "content": prompt}],
            max_tokens=800,
        )
        return r.choices[0].message.content
    except openai.APITimeoutError:
        print("timed out after retries")
    except openai.RateLimitError as e:
        print("429, slow down:", e.request_id)
    except openai.APIConnectionError:
        print("cannot reach the API")
    except openai.APIStatusError as e:
        print("HTTP", e.status_code, e.request_id)
    return None

The order is narrow to broad. APIStatusError covers any 4xx or 5xx response, so if it came first it would swallow RateLimitError. Same idea for the timeout: it sits above the general connection error. In a batch script, return None is the whole point: log the prompt, move on to the next one, and rerun the failures at the end.

When the timeout fires and the retries are used up, this is what the exception looks like:

For the 429 case in detail, including waiting between tries, see 429 Too Many Requests from an AI API.

Cap max_tokens so a long answer cannot run away#

A timeout stops waiting. It does not make the model write less. Without max_tokens, a model with a 32,768-token answer limit is free to write a very long reply, and a long reply is a slow one. Set a limit that fits the job: 800 for a ticket summary, 5 for a yes/no.

The cost is that an answer can be cut off. Check resp.choices[0].finish_reason: length means the cap hit before the model finished. Why does an AI answer stop mid-sentence? covers it. If you stream the answer instead, the first words arrive early and the wait feels shorter, see Streaming responses in Python.

What retries do to the bill#

A retry is a new request. If an attempt was processed, it is a separate request with its own tokens. Say a ticket summary is 1,500 tokens in and 400 out: one request costs $0.000063 at today's prices, and the full pricing table has the other models. Prices are fixed per model, with no peak hours, so a retry at 3 a.m. costs the same as one at noon.

Move the sliders to your own batch:

The table shows one pass. If a tenth of the batch needs a second attempt, add a tenth. Retries are cheap insurance when failures are rare, and an expensive habit when you set max_retries=10 against an endpoint that keeps failing.

Before you ship#