Stream failed after HTTP 200: catch mid-stream errors in Python

Your chat UI streamed half an answer, froze, and the log says 200 OK. That is not a contradiction: the 200 only says the stream started, not that it finished. If something breaks later, the server can only send an error inside the stream, so a check on the status code never sees it. The fix is to treat a stream as successful only when it ends with its finish signal, and to retry from a clean buffer when it doesn't.

The rest of this page shows what the broken stream looks like, two Python handlers (Anthropic Messages and OpenAI SDK), and a retry that doesn't print the beginning of the answer twice. Both handlers use CheapTokens as the API endpoint: the docs have the base URLs, and the sample model is DeepSeek-V4-Flash (id deepseek-ai/DeepSeek-V4-Flash-0731, 380K context, up to 32,768 output tokens per answer, one price at any hour — see the model page).

What a 200 really promises#

Think of a courier who signs for a parcel at the door: the signature proves he arrived, not that the parcel is intact. HTTP works the same way. The server sends the status line and headers first, then the body arrives piece by piece. If the model backend gets overloaded halfway through, the status line is already gone.

Anthropic's API errors page (checked 1 November 2026) says it directly: when you receive a streaming response over SSE, an error can occur after the API returns a 200, and then the standard error handling does not apply. Its streaming docs give the shape of that event:

event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}

overloaded_error normally corresponds to HTTP 529 on a non-streaming call (temporary overload from high traffic). The same page lists 500 api_error (retry with exponential backoff) and 504 timeout_error (use streaming for long requests). On the wire a broken stream looks like this:

The docs also warn that new event types may be added, so your code should handle unknown events gracefully. That's why the handlers below don't try to recognise every error byte. They ask one question: did the stream end with its finish signal?

• Anthropic Messages: the last event is message_stop, and stop_reason in message_delta says why it ended (end_turn, max_tokens, stop_sequence).
• OpenAI Chat Completions: the last chunk carries a non-empty finish_reason (stop, length, …).

No finish signal, or an exception on the way — the answer is incomplete, whatever the status said.

Catch it in Python#

Take a support bot that streams replies to the browser. The loop collects text into a buffer and reports back three things: the text so far, whether the stream finished, and the error if there was one. Pick the tab for the SDK you use.

import os
import anthropic

client = anthropic.Anthropic(
    base_url='https://api.cheaptokens.dev',  # no /v1 for the Anthropic SDK
    api_key=os.environ['CHEAPTOKENS_API_KEY'],
)
MODEL = 'deepseek-ai/DeepSeek-V4-Flash-0731'

def ask_once(prompt, on_text=lambda t: None):
    parts, finished = [], False
    try:
        with client.messages.stream(
            model=MODEL,
            max_tokens=1024,
            messages=[{'role': 'user', 'content': prompt}],
        ) as stream:
            for event in stream:
                if event.type == 'content_block_delta' and event.delta.type == 'text_delta':
                    parts.append(event.delta.text)
                    on_text(event.delta.text)
                elif event.type == 'message_stop':
                    finished = True
    except anthropic.APIError as err:
        return ''.join(parts), False, err
    return ''.join(parts), finished, None

import os
import openai

client = openai.OpenAI(
    base_url='https://api.cheaptokens.dev/v1',
    api_key=os.environ['CHEAPTOKENS_API_KEY'],
)
MODEL = 'deepseek-ai/DeepSeek-V4-Flash-0731'

def ask_once(prompt, on_text=lambda t: None):
    parts, finish = [], None
    try:
        stream = client.chat.completions.create(
            model=MODEL,
            max_tokens=1024,
            stream=True,
            messages=[{'role': 'user', 'content': prompt}],
        )
        for chunk in stream:
            if not chunk.choices:
                continue
            choice = chunk.choices[0]
            if choice.delta.content:
                parts.append(choice.delta.content)
                on_text(choice.delta.content)
            if choice.finish_reason:
                finish = choice.finish_reason
    except openai.APIError as err:
        return ''.join(parts), False, err
    return ''.join(parts), finish is not None, None

Two details are worth a second look. First, both handlers catch the SDK's base error class (anthropic.APIError, openai.APIError), so it doesn't matter which subclass the SDK raises for a given failure. Second, the finished flag is separate from the exception: a stream can also just end early, with no exception at all, and that case must count as a failure too.

The openai-python README adds what to expect on the OpenAI side (checked 1 November 2026): a read timeout in a stream raises APITimeoutError, other HTTPX failures raise APIConnectionError with the original exception in __cause__. If the stream stalls instead of failing, see timeouts and retries in the OpenAI Python SDK.

Retry without printing the answer twice#

Here is a bug that is easy to ship. A first attempt prints "The refund policy is…", dies, the retry starts over and prints "The refund policy is…" again — now the user sees the beginning twice, or a sentence glued to its own restart. The usual cause is one shared buffer: answer += text lives outside the attempt, so the second try appends to the first.

The cure is boring. One attempt owns one buffer, a failed attempt's text is thrown away (or kept only for the log), and the retry starts from scratch:

import time

def ask_with_retry(prompt, attempts=3, on_text=lambda t: None, on_restart=lambda: None):
    for attempt in range(attempts):
        text, finished, err = ask_once(prompt, on_text)
        if finished:
            return text
        status = getattr(err, 'status_code', None)
        if status is not None and 400 <= status < 500 and status not in (408, 409, 429):
            raise err  # a 400/401/404 will not fix itself
        print(f'attempt {attempt + 1} failed after {len(text)} chars: {err!r}')
        if attempt == attempts - 1:
            break
        on_restart()  # clear the message box in your UI
        time.sleep(2 ** attempt)  # 1 s, 2 s
    raise RuntimeError('stream failed after all attempts')

on_restart is the part that stops the duplicate. In a web UI it empties the message bubble before the next attempt streams in. In a plain terminal you can't un-print, so either print only after finished is true, or print a visible --- retrying --- line. Retrying from scratch resends the whole prompt, so the input is billed again for the new attempt; for most chat replies that's a fair price for a clean answer.

Anthropic's SDK already retries transient failures — connection errors, rate limits, 5xx — with exponential backoff, 2 times by default, honouring retry-after, and max_retries changes that. That covers the request. A stream that dies after the 200 is a different story: the docs say standard error handling does not apply there, so the loop above is yours to keep. For 429 on its own, there's a separate walkthrough: 429 Too Many Requests: retries with backoff.

Not every early stop is an error#

Before you retry, check why the answer ended. A stream that finishes with stop_reason: max_tokens (Anthropic) or finish_reason: length (OpenAI) did finish — it just hit the limit you set. Retrying that returns the same cut-off text and the same bill. Raise the limit instead; the cut-off answer guide walks through it.

There is a third group: the network. Idle connections may be dropped on the way, which shows up as a connection error or a stream that stops without any event. Handle it the same as a mid-stream error, and see common connection errors if it keeps happening. For long generations, Anthropic recommends streaming over a huge non-streaming max_tokens, so you're already on the right side of that.

Before you ship#

Save the request id of every failed attempt. Anthropic error bodies carry a request_id field and a request-id response header, and the Python SDK exposes message._request_id; with that id a failure is traceable instead of anecdotal.

If you want to see the handler run, point it at your own key: create an API key in the docs, set CHEAPTOKENS_API_KEY, and call ask_with_retry('Explain SSE in two sentences'). For the basics of streaming in Python, start with streaming responses in Python; the Anthropic side is covered in the Anthropic Python SDK with CheapTokens.