Thinking tokens in a bulk job: effort, caps, stage budgets

Thinking tokens are billed as output tokens, and they count toward max_tokens even when you never see them. So the bill of a big job depends on how much the model thinks, not only on how long its visible answer is. The two levers that matter are an output cap and a split of the job: a cheap model for the mass pass, a stronger one only for assembly.

If your agent ran for ten minutes and the bill is bigger than the task, this is the first place to look.

What a thinking token looks like on the bill#

Take a classifier that answers keep or drop for each comment. The visible answer is a single word; let's budget about 20 output tokens for it. Say the model reasons first and spends 600 tokens doing it. You see 20 tokens on screen and pay for 620.

Anthropic's effort page (checked 5 November 2026) is explicit about it: thinking counts toward max_tokens even when the thinking content is not returned, and max_tokens is a hard limit on the total output, thinking plus response text. The output side is also the side that is priced higher per token, which is why this hurts.

Multiply that by 40,000 comments and the invisible part is most of the invoice. For the basics of the two sides of a request, see input vs output tokens.

Effort: a dial, not a budget#

Anthropic's Messages API sets effort with output_config.effort. Without a value, the model's default applies. The levels are max, xhigh, high, medium and low. The default is high on most models; Claude Opus 5.5 and Claude Haiku 5.5 default to medium.

Three details from the same page are worth keeping in mind:

• Effort applies to every output token, with or without thinking. Lower effort also means fewer and terser tool calls, which matters for agents.
• low suits simpler tasks and subagents.
• Effort is a behavioural signal, not a strict token budget. It nudges the model; it does not promise a number. The hard limit is max_tokens.

So effort is the dial for how hard the model tries, and the output cap is the fence. You want both where they are available. For the plain story of why thinking costs more, there is a beginner version: reasoning effort and thinking tokens explained.

A real pipeline: one model directs, another does the bulk#

Andrej Karpathy, AI researcher and OpenAI co-founder, described his HN video pipeline on 31 October 2026 (the post). Claude Opus 5.5 at Medium effort, its default, downloaded all the Hacker News comments, narrowed them to about 40,000 comments that recommend a YouTube video, and built a site with 6,454 videos from 2007 to 2026 across 150 topics. Sonnet ran over the comments in a Claude Code workflow to filter spam and find the recommendations. By his account the whole thing takes about two hours now.

The post gives no token counts and no cost, so there is nothing to quote on price. What it does show is the shape: the stronger model sits at the top at medium effort and decides what to build; a cheaper model grinds through the repetitive part. Note where the tokens go. Directing is a few big, rare steps. Filtering is tens of thousands of tiny, identical ones.

The same split works with any pair of models. If you want the filter prompt itself, it is in filtering 40K comments with an LLM; for deciding which request goes to which model, see model routing, cheap first.

Estimate the bulk pass#

Here is our own assumption, not Karpathy's numbers: 40,000 comments, about 150 input tokens each (instruction plus comment) and about 20 output tokens each (a label and a few words).

• Input: 40,000 × 150 = 6,000,000 tokens
• Output: 40,000 × 20 = 800,000 tokens

On DeepSeek-V4-Flash that pass costs $0.17. The table shows the same workload on every model we serve; in total, with input and output counted together, each one is cheaper than OpenRouter.

Now let the model think for 600 tokens per comment before it answers, so 620 output tokens per item: 40,000 × 620 = 24,800,000 output tokens. The same pass on DeepSeek-V4-Flash becomes $2.57. Same input, and if the model thinks 600 tokens per comment, the output side grows thirty-one times. Whether that thinking is worth it is for your own eval to say; for a one-word keep/drop it is the first thing to test at a lower level or with a cap.

Play with your own numbers: tokens per request and number of requests, and the calculator uses live prices.

Levers you can pull today#

Different APIs expose different thinking controls, so here are the ones that work everywhere: a hard cap, a short-answer instruction and a stage split.

A minimal request for the mass pass on CheapTokens. The base URL is https://api.cheaptokens.dev/v1 and the model id is the full one from the docs:

curl https://api.cheaptokens.dev/v1/chat/completions \
  -H "Authorization: Bearer $CHEAPTOKENS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V4-Flash-0731",
    "max_tokens": 1024,
    "messages": [
      {"role": "system", "content": "Reply with exactly one word: keep or drop."},
      {"role": "user", "content": "Comment: this talk on compilers changed how I think, youtube.com/watch?v=example"}
    ]
  }'

Read usage.completion_tokens in the response: that number is your output bill for the request. The cap fences thinking and the visible answer together. If finish_reason is length and the content is empty, the cap was spent before the answer appeared: raise it, or ask for a shorter answer. Check usage.completion_tokens on a few comments, then set the cap just above the real maximum. The cap-and-stop-sequence details are in cut output tokens, and the mid-sentence cut is explained in why an answer stops mid-sentence.

DeepSeek-V4-Flash allows up to 32,768 output tokens per answer with a 380K context, so a cap of 1,024 is a choice, not a limit. GLM-5.3-Flash (8,192 output, 390K context) and MiniMax-M2.7 (8,192 output, 180K context) fit the same pattern. Pick the cap per stage.

To see the whole thing on one page, the live price table is at CheapTokens pricing, and the request format is in the docs.

Effort per stage: a short rule#

Start low on the mass pass and raise one stage at a time. Keep the number that your own eval says is worth it, not the one that sounds safest. The strong model should see a small, already filtered set; the cheap one should see everything else.