Model routing to cut API cost: cheap model first, escalate

Routing can cut the bill when the cheap model's first pass is much cheaper than the second model and few requests escalate: send every request to the cheap model first and re-send only the hard ones to a second model. Whether you actually save depends on the share of requests you escalate. Below is the arithmetic for 10,000 requests at 10%, 30% and 50%, and a Python function you can paste into your project.

The pattern is not new. Anthropic's engineering post "Building effective agents" (19 December 2024) calls it a routing workflow: common, easy questions go to a smaller, cost-efficient model, and unusual, hard ones go to a more capable one. The post gives no prices, so the numbers here are our own, on live CheapTokens prices.

Which of your requests are easy?#

Routing only works if you can say, in a rule, what "hard" means. Not in a feeling, in a rule the code can check. Pick the case closest to yours:

Do not escalate on "the model sounds unsure" unless you told it how to say so. A one-line instruction ("reply exactly UNSURE if you cannot answer") turns a vibe into something if can read.

What 10,000 requests cost: cascade vs router#

Say you run a support bot with 10,000 requests a month. An average request is 800 tokens in and 300 tokens out. These are assumptions; swap in your own from usage (our token count article shows where to read them).

There are two ways to wire routing, and they bill differently:

• Cascade. Every request goes to the cheap model. A share p fails the check and is sent again to the second model. Those requests are paid twice: cost = N × [cheap + p × second].
• Router. A small classifier call decides up front. Easy ones go to the cheap model, hard ones straight to the second one: cost = N × [(1 − p) × cheap + p × second], plus the classifier on every request.

We will use the cascade, because it needs no classifier and is the easier one to start with. The first pass is GLM-5.3-Flash (zai-org/GLM-5.3-Flash, 390K context, up to 8,192 output tokens). The second pass is DeepSeek-V4-Flash release 0731 (deepseek-ai/DeepSeek-V4-Flash-0731, 380K context, up to 32,768 output tokens), so an answer that did not fit in 8,192 tokens has room to finish. Both are cheaper than OpenRouter in total, with input and output counted together; the live table below shows the OpenRouter column next to ours.

The first pass always covers all 10,000 requests: 8,000,000 input and 3,000,000 output tokens. The second pass covers only the escalated share. The last column is the alternative: all 10,000 requests straight on the second model.

Add the first-pass and second-pass columns in each row to get the cascade's month, then compare that sum with the last column. The first pass is a fixed cost, and only the second pass grows with p. That is the whole story of a cascade: the more you escalate, the closer you get to paying for the work twice.

Here is the same load for every model at once, so you can see where the single-model options land:

When is a cascade not worth it? When the first pass plus the escalated share costs more than sending everything to the second model. In symbols: cheap + p × second < second, which means p < 1 − cheap/second. For this pair, the first pass costs a noticeable share of the second model's price on this load, so the gain from a cascade shrinks, and at a high escalation share it disappears. At 50% the cascade can lose to simply sending all 10,000 requests to the second model, so check the sum of the two passes against the last column before you commit. The ratio depends on your input/output mix, because the two models price the two sides differently, so read it from your own row.

The router variant adds one thing: the classifier. Assume it reads 300 input tokens and answers in 5 tokens on the cheap model, per request. For 10,000 requests that is 3,000,000 input and 50,000 output tokens, or $0.062 a month on top. Compare this line with the share p of the first-pass column (10%, 30% or 50% of it): that is what the cascade wastes by running the first pass on requests that then escalate. If the classifier line is larger, the cascade is the cheaper start; if it is smaller, a router saves more.

Try your own numbers before you build anything:

Change runs to your volume and read the total for each model. Then use the table above to see how much of it a second pass would add.

A router in Python: two calls and one check#

Both models sit behind one OpenAI-compatible base URL and one balance, so routing needs no second account or key. The code below is an example: the numbers in it (max_tokens, the UNSURE rule) are yours to tune. The second call gets a higher max_tokens, so an answer cut off at 600 tokens can finish. Base URL, model ids and the rest are in the CheapTokens docs.

import os
from openai import OpenAI

client = OpenAI(
    base_url='https://api.cheaptokens.dev/v1',
    api_key=os.environ['CHEAPTOKENS_API_KEY'],
)

CHEAP = 'zai-org/GLM-5.3-Flash'
SECOND = 'deepseek-ai/DeepSeek-V4-Flash-0731'
SYSTEM = 'Answer briefly. If you cannot answer well, reply exactly UNSURE.'


def ask(model, question, max_tokens):
    r = client.chat.completions.create(
        model=model,
        max_tokens=max_tokens,
        messages=[
            {'role': 'system', 'content': SYSTEM},
            {'role': 'user', 'content': question},
        ],
    )
    choice = r.choices[0]
    return choice.message.content or '', choice.finish_reason


def needs_escalation(text, finish_reason):
    return finish_reason == 'length' or not text.strip() or 'UNSURE' in text


def answer(question):
    text, why = ask(CHEAP, question, 600)
    if needs_escalation(text, why):
        text, why = ask(SECOND, question, 4000)
        if 'UNSURE' in text:
            text = 'Sorry, I could not answer this one.'
    return text

Three habits make this honest rather than just cheap:

• Count the escalations. Log how often needs_escalation fires. That is your p, and it is the number that decides whether the cascade pays.
• Choose the second model on your own test set. Take 50 hard requests from your logs, run them through both models and keep the second model only if it fixes the ones the first one missed. Its documented edge is the 32,768-token answer limit, and your own set shows whether that is the edge you need.
• Send the same request, not a longer one. If you paste the failed answer into the second call, you pay for it as input. Start the second pass clean unless the first answer is genuinely useful context.

Check yourself#

Where to go next#

If you are not sure which model should take the first pass, MiniMax-M2.7 vs DeepSeek-V4-Flash compares them by task. For a per-task view instead of a per-token one, see price per 1M tokens vs cost per task, and for loops with many steps, how to estimate the cost of an AI agent. Live prices for every model are on the pricing page, and the model pages are DeepSeek and GLM.