Trim chat history to cut cost: sliding window or summary

Your support bot has been chatting for 50 turns, and every new message costs more than the one before. The API remembers nothing, so your code resends the whole conversation each time. The fix is to cap what you send: keep the system prompt and the last few turns (a sliding window), or swap old turns for a short summary. Below is the arithmetic for a 50-turn chat and a helper you can copy.

DeepSeek's multi-round chat guide (api-docs.deepseek.com, checked 1 November 2026) says it plainly: the server does not record the context of earlier requests, so the client sends the full messages list on every call. The OpenAI-compatible format on CheapTokens works the same way. If that is new to you, start with why a long chat costs more and how the messages array works.

What turn 50 actually sends#

Say you run a support bot with a 400-token system prompt. In a typical turn the user writes about 60 tokens and the bot answers with about 190. One full turn adds 250 tokens to the history. Those are assumptions: replace them with the numbers from the usage field of your own responses.

On turn 50 the request carries the system prompt, 49 earlier turns and the new message: 400 + 49 × 250 + 60 = 12,710 input tokens, to get a 190-token answer. Add up the whole chat. The system prompt and the new message are sent every time: 50 × 460 = 23,000. The repeated history adds 250 × 1,225 = 306,250. Together: 23,000 + 306,250 = 329,250 input tokens, against 50 × 190 = 9,500 output tokens.

The history in each request grows by one turn, so the total grows with the square of the chat length. Go from 50 to 100 turns and the repeated-history part roughly quadruples.

Option 1: a sliding window#

The idea: always send the system prompt, drop everything except the last N turns. You keep the full history in your own storage and trim only the copy you send.

def trim_history(messages, keep_pairs=4):
    """System prompt + the last keep_pairs turns + the new user message."""
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] != "system"]
    return system + rest[-(keep_pairs * 2 + 1):]

Call it after you append the user's new message. With keep_pairs=4 that is 4 user/assistant pairs plus the new message, 9 messages, and the slice always starts on a user message. Here it is wired into a real call:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.cheaptokens.dev/v1",
    api_key=os.environ["CHEAPTOKENS_API_KEY"],
)
MODEL = "deepseek-ai/DeepSeek-V4-Flash-0731"

history = [{"role": "system", "content": "You are a concise support assistant."}]

def ask(text):
    history.append({"role": "user", "content": text})
    resp = client.chat.completions.create(
        model=MODEL,
        messages=trim_history(history),
        max_tokens=500,
    )
    answer = resp.choices[0].message.content
    history.append({"role": "assistant", "content": answer})
    print(resp.usage.prompt_tokens, resp.usage.completion_tokens)
    return answer

The print line is the useful one: watch the first number. Without trimming it climbs every turn. With the window it stops growing once the chat passes turn 4.

In numbers: from turn 5 on, each request is 400 + 4 × 250 + 60 = 1,460 input tokens. The first four turns still grow normally and add up to 3,340. The whole chat: 3,340 + 46 × 1,460 = 70,500 input tokens, about a fifth of the 329,250 you send with full history. Output stays 50 × 190 = 9,500.

In code, the whole change is one argument in the call:

messages=history

messages=trim_history(history)

The catch: a window forgets. If the user asks on turn 40 "what was my order number?" and the number was in turn 3, the model has never seen it. Two cheap fixes. Pull stable facts (name, order number, plan) out of the chat in your own code and append them to the system prompt, where the window never cuts them. Or add a summary, which is the next option.

Option 2: a summary of old turns#

Instead of dropping old turns, you compress them. Every few turns, send the turns that just fell out of the window, plus the current summary, to the model and ask for a new, short summary. Then put that summary into the system message.

def summarize(summary, old_messages):
    transcript = "\n".join(f"{m['role']}: {m['content']}" for m in old_messages)
    prompt = (
        f"Summary so far:\n{summary}\n\nNew turns:\n{transcript}\n\n"
        "Rewrite the summary in under 200 words. "
        "Keep names, numbers, decisions and open questions."
    )
    resp = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=400,
    )
    return resp.choices[0].message.content

def build_messages(history, summary, keep_pairs=4):
    system = history[0]["content"]
    if summary:
        system += "\n\nEarlier in this chat: " + summary
    recent = history[1:][-(keep_pairs * 2 + 1):]
    return [{"role": "system", "content": system}] + recent

The summary is not free: it is an extra model call. Assume a summary of about 300 tokens, refreshed every 5 turns (9 times in 50 turns), each refresh reading about 1,600 tokens (the old summary plus five turns plus the instruction) and writing 300. To stay on the safe side, count the summary in every request from turn 5 on (in reality it appears after the first refresh, so this is a slight overestimate): 400 + 300 + 4 × 250 + 60 = 1,760 input tokens.

Totals for the chat: main requests 3,340 + 46 × 1,760 = 84,300 input tokens, summary calls 9 × 1,600 = 14,400 input and 9 × 300 = 2,700 output. Altogether 84,300 + 14,400 = 98,700 input and 9,500 + 2,700 = 12,200 output tokens.

The summary costs a bit more than the bare window, and in return the model still knows what happened in turn 3.

What the three setups cost#

Here is the same 50-turn chat on DeepSeek-V4-Flash, priced live from the totals above:

The prices come from the live price list, so the numbers update when the tariff does. The ordering will not: the window sends the least, the summary sits in between, full history is the most expensive.

For a chat like this, the CheapTokens model to start with is DeepSeek-V4-Flash. It has a 380K context window (a 50-turn chat fits even untrimmed), up to 32,768 output tokens per answer, one fixed price at any hour of the day, and in total, with input and output counted together, it is cheaper than OpenRouter. The model id is deepseek-ai/DeepSeek-V4-Flash-0731. If you want to compare models on the window setup (1,410 input tokens per request on average, 190 output, 50 requests), here is the live table:

GLM-5.3-Flash has a 390K context and MiniMax-M2.7 has 180K, which is enough for a trimmed chat. Current prices are on the pricing page, and the docs show how to send the first request.

Which one to pick#

A window is the default: three lines of code, no extra calls, no extra failure mode. Use it for support bots, tutors and role-play, where the last few turns carry the conversation and the system prompt carries the rules. Add the summary when users refer back to things said long ago, as in a personal assistant with one never-ending conversation. Keep the window small enough that the model still sees the immediate context, and big enough to hold a full question-and-answer exchange. How much fits at all is a separate limit, covered in what a context window is.