What a RAG question costs: how many chunks to send

Your document chatbot works, the first invoice arrives, and it is bigger than the questions deserve. In almost every RAG setup the money goes to one place: the retrieved chunks you paste in front of every question. One RAG question costs the tokens of the question, your instructions and N chunks on the input side, plus a short answer on the output side. Chunks are the part you control, and each one adds the same number of tokens to every single request.

This guide counts one request, prices 1,000 and 100,000 of them on live prices, and shows what changes when you move top-k from 3 to 10. Retrieval itself (your vector store, your chunking) stays on your side; only the final generation call goes to the model API. If input and output billing is new to you, start with input vs output tokens.

What goes into one RAG request#

Say you run a support bot over company docs. A customer asks, "How do I reset two-factor login?", your retriever returns the five best passages, and you send this to the model:

• a system prompt: "Answer only from the context. If it is not there, say so." — about 300 tokens once you add formatting rules;
• the question with a bit of chat context — about 200 tokens;
• 5 chunks of about 500 tokens each — 5 × 500 = 2,500 tokens;
• the answer: two or three short paragraphs — about 300 tokens out.

The input side is 2,500 + 300 + 200 = 3,000 tokens, the output side about 300. These are the author's assumptions for a typical support bot; swap in your own numbers. Everything below uses this one request as the unit.

The quick way to see the shape: the chunks are roughly five sixths of the whole input, and the input is ten times the answer. Try the estimate yourself:

What 1,000 questions cost#

One request is too small to feel, so scale it. With 3,000 tokens in and 300 out, 1,000 questions send 3,000,000 input tokens and generate 300,000 output tokens. Drag the sliders below to match your own bot; the site recalculates on live prices for every model.

The table shows the same workload for all three models, with the OpenRouter reference next to it. Counted in total, with input and output together, each of them comes out cheaper than OpenRouter here.

And at 100,000 questions? That is 300,000,000 tokens in and 30,000,000 out. On DeepSeek-V4-Flash that is $7.50, on GLM-5.3-Flash $7.20, on MiniMax-M2.7 $19.50. The prices are fixed per model, with no peak hours, so a night batch and a daytime chat cost the same per token. See the full price list on the pricing page.

Top-k: every chunk adds the same amount#

Here is the second scenario: an internal wiki assistant. Somebody suggests, "Let's send 10 chunks instead of 3, just in case." The input grows in a straight line: each chunk is another 500 tokens in every request.

2,000 against 5,500 tokens: the answer is the same length, so the whole difference in the wiki assistant's bill is input. Whether it buys you a better answer is a quality question: past the point where the right passage is already in the context, extra chunks mostly add noise and cost. A practical way to pick k is to take a set of real questions, check at which k the right passage shows up, and stop a little above that.

Context window is the other limit. All three models here have plenty of room for ordinary top-k: 180K tokens for MiniMax-M2.7, 380K for DeepSeek-V4-Flash and 390K for GLM-5.3-Flash. Output limits differ: up to 8,192 tokens per answer for GLM-5.3-Flash and MiniMax-M2.7, up to 32,768 for DeepSeek-V4-Flash, which matters if your answers are long reports rather than short replies.

The whole document in the prompt, or four chunks#

Third scenario: contract Q&A. The lazy version pastes the entire 30,000-token PDF into every question. The retrieval version sends the question, the instructions and four relevant chunks of about 650 tokens each, roughly 3,000 tokens in total. Same 1,000 questions, same 300-token answers:

The full document sends ten times the input, 30,000 ÷ 3,000 = 10, for the same question. It still makes sense for a single one-off question about a document that fits in context. For a bot that answers all day, it is the expensive default. If a lot of that input is the same text every time, read about prompt caching.

Send the request#

Below is the generation step. The retrieval part is yours: chunks is whatever your vector store returned. The same code works with the OpenAI SDK or plain curl.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.cheaptokens.dev/v1",
    api_key="YOUR_CHEAPTOKENS_KEY",
)

chunks = ["Chunk 1 text...", "Chunk 2 text...", "Chunk 3 text..."]
context = "\n\n---\n\n".join(chunks)

resp = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash-0731",
    max_tokens=400,
    messages=[
        {"role": "system", "content": "Answer only from the context. If the answer is not there, say so."},
        {"role": "user", "content": f"Context:\n{context}\n\nQuestion: How do I reset two-factor login?"},
    ],
)
print(resp.choices[0].message.content)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)

curl https://api.cheaptokens.dev/v1/chat/completions \
  -H "Authorization: Bearer YOUR_CHEAPTOKENS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V4-Flash-0731",
    "max_tokens": 400,
    "messages": [
      {"role": "system", "content": "Answer only from the context. If the answer is not there, say so."},
      {"role": "user", "content": "Context:\nChunk 1...\n\nQuestion: How do I reset two-factor login?"}
    ]
  }'

The last line of the Python example is the useful one. usage.prompt_tokens is your real input per question; compare it with the 3,000 estimate and replace my assumptions with your numbers. Use the full model id, as in the example: deepseek-ai/DeepSeek-V4-Flash-0731. The ids are listed in the docs. Full setup is in the docs, and if you want to see how a bot's tokens add up over a dialogue, why a long chat costs more covers the history part.

Four ways to shrink the input#

Start with max_tokens, which caps the answer and protects you from a runaway reply. Then cut k to the smallest value that still finds the right passage. Make chunks tighter, so 500 tokens carry 500 tokens of meaning and not headers and boilerplate. Drop duplicate chunks before sending: two near-identical passages cost twice and say once. And measure with usage on real traffic, because a guess about 500-token chunks is only a guess. For a wider look at where tokens go in an API call, see how many tokens a request uses.

Put your own tokens per request into the calculator above to price your bot on all three models; if your answers are long, DeepSeek-V4-Flash is the one to look at first.