Why a long AI chat costs more: the API resends history

The first messages of a chat are cheap. Message fifty costs noticeably more than message ten, even though you typed the same amount. Nothing is broken: this is how chat APIs work.

The short answer#

A model does not remember your previous requests. OpenAI's documentation puts it directly: "each text generation request is independent and stateless", and for a multi-turn conversation you include the earlier messages and the model's previous answer in the next request. Anthropic's Messages API describes the same pattern: stateless multi-turn conversations, where earlier turns are passed in the messages parameter.

So the chat window you see in an app is an illusion built by the app. Behind it, every new message is a fresh request that carries the whole conversation so far. You pay for input tokens on every request, and the history is part of that input. If you are new to the terms, read What is a token and Input vs output tokens first.

OpenAI states the billing side explicitly for its response chains: all previous input tokens in the chain are billed as input tokens. The same principle holds for any API where you send the history yourself: what you send is what is counted.

A worked example with made-up numbers#

The numbers below are a teaching example, not a measurement. Assume each of your messages is about 200 tokens and each answer is about 300 tokens. One finished exchange adds 500 tokens to the history.

The answer stays the same length, but the input of turn 20 is almost fifty times the input of turn 1. Output does not grow; input does.

Now the total for a 20-turn chat. The history is resent 0, 1, 2 ... 19 times as many exchanges, which is a simple sum:

• Old exchanges sent again: 500 × 190 = 95,000 tokens (190 is 0 + 1 + 2 + ... + 19).
• New messages: 200 × 20 = 4,000 tokens.
• Total input for the chat: 95,000 + 4,000 = 99,000 tokens.
• Total output: 300 × 20 = 6,000 tokens.

If the model remembered everything for free, the input would be only 200 × 20 = 4,000 tokens. The rest is the history being sent again. Doubling the number of turns does not double the cost of the history, it roughly quadruples it, because the sum grows with the square of the turn count.

What it costs on live prices#

Here is the same 20-turn chat (99,000 input, 6,000 output) at current prices. The table is live and updates when tariffs change; the last column is OpenRouter's reference price for comparison.

For a quick check in text: on MiniMax-M2.7 this whole conversation costs $0.0059. For a chat like this, where input dominates, the input price matters most: on MiniMax-M2.7 it is cheaper than OpenRouter's reference price. Per 1M tokens, MiniMax-M2.7 is $0.05 for input and $0.15 for output.

Three ways to keep it down#

1. Start a new chat when the topic changes#

The cheapest history is an empty one. If you finished debugging a function and now want a recipe, a new chat sends 200 tokens instead of 9,700. In an app this is one button; in code it is history = []. Do it whenever the earlier context no longer helps the next question.

2. Trim the history#

Keep only the last few messages and drop the rest. Suppose you keep the last three exchanges, so the history never exceeds 500 × 3 = 1,500 tokens:

• Turns 1-3 grow normally: 0 + 500 + 1,000 = 1,500 tokens of history.
• Turns 4-20 each carry the capped 1,500: 1,500 × 17 = 25,500 tokens.
• History total: 1,500 + 25,500 = 27,000 tokens, plus 4,000 of new messages = 31,000 tokens of input.

That is 31,000 instead of 99,000 for the same 20 turns, and on MiniMax-M2.7 it comes to $0.0025. The price of trimming is that the model forgets the early part of the chat, so keep the system prompt and any facts it must not lose outside the trimmed part.

3. Replace old turns with a summary#

When the history gets long, ask the model once to summarise it, then continue with the summary instead of the full text. In the example above, nine exchanges are 4,500 tokens; a summary of about 300 tokens carries the main points forward. You pay for one extra request to write the summary, and every later request is lighter. This keeps more meaning than plain trimming, at the cost of a bit of detail.

The context window is a separate limit: it caps how many tokens one request may hold, including the answer. The history will hit that ceiling eventually; Why does an AI answer stop mid-sentence? explains it.

See it in your own code#

The response contains a usage object, so you can watch the growth yourself. This is an example: replace the key and take the model id from the CheapTokens documentation, because ids differ between providers.

from openai import OpenAI

client = OpenAI(base_url='https://api.cheaptokens.dev/v1', api_key='YOUR_KEY')

system = {'role': 'system', 'content': 'You are a helpful assistant.'}
history = []
KEEP = 7  # new message + last 3 exchanges (6 messages); None keeps everything

def ask(text):
    history.append({'role': 'user', 'content': text})
    r = client.chat.completions.create(
        model='MODEL_ID_FROM_DOCS',
        messages=[system] + (history[-KEEP:] if KEEP else history),
    )
    answer = r.choices[0].message.content
    history.append({'role': 'assistant', 'content': answer})
    print(r.usage.prompt_tokens, r.usage.completion_tokens)
    return answer

Call ask() several times in a row and print prompt_tokens: with KEEP = None it climbs every turn, with KEEP = 7 it levels off. For agents that loop through many steps the same effect applies to every step; How to estimate the cost of an AI agent shows how to budget for it.

Where to run long chats#

Long dialogues and agent loops are exactly the workloads where input volume dominates, so a large context window and room for long answers both matter:

• MiniMax-M2.7: 180K context, up to 8,192 tokens per answer, input and output both cheaper than OpenRouter's reference price.
• DeepSeek-V4-Flash: 380K context, up to 32,768 tokens per answer, output cheaper than OpenRouter's reference price.

CheapTokens has one balance for all models, a fixed price per model with no peak hours, and an API compatible with OpenAI Chat Completions and Anthropic Messages, so the code above works by changing the base URL. Top up the balance and send your first request by following Your first API request with CheapTokens.