Nemotron Bolt E4B or a hosted API? Choose by context and task

If your task fits inside 64,000 tokens and your GPU has room for a ~4 GB model, NVIDIA's Nemotron 3.5 Bolt E4B can run it on your own machine. If the job needs a bigger window, a long answer, or 20,000 items finished today, a hosted model is the shorter road. Below is a table that sorts your task, and the arithmetic behind each row.

The hosted side of the comparison uses the three models on the CheapTokens models page: DeepSeek-V4-Flash, GLM-5.3-Flash and MiniMax-M2.7.

What NVIDIA published, and what it left out#

NVIDIA's research page for Nemotron 3.5 Bolt E4B (released 23 October 2026, checked 5 November) describes a model built on Gemma 4 E4B and shipped as a GGUF checkpoint, Q4_K_M quantization, for llama.cpp. NVIDIA says it fits in as little as 4 GB; the VRAM it measured ran from 3.1 to 5.0 GB (on NVIDIA's test workload; with a full 64K window memory use will be higher, so check on your own card). The weights are on Hugging Face as nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB and also come through Ollama, LM Studio and Unsloth.

Three facts decide most choices:

• Context: 64K tokens. NVIDIA is explicit that the window holds everything at once: chat history, instructions, tool definitions and retrieved text.
• Languages: 35, from a base model pre-trained on more than 140.
• Speed: in NVIDIA's llama.cpp CUDA runs (Q4_K_M, batch size 1) an RTX 5070 with 12 GB used 3.5 GB of VRAM and produced 133.4 tokens per second. A Jetson Orin Nano with 8 GB used 4.2 GB and produced 15.5 tokens per second. Both numbers come from NVIDIA's own test workload, so expect yours to differ.

The page also lists tool-use scores (BFCL-v3 68.5, Tau2-Bench 55.1) and RULER at 64K of 87.0. These are the developer's own benchmarks: they say what the model was tested on, not how it stacks up against a hosted model, so this article doesn't line them up.

Two things the page doesn't state: how long a single answer can be, and the licence. Read both on the Hugging Face model card before you build anything on it.

How big is 64K when an agent starts working?#

Think of the window as a desk. The model can only work with what is lying on it right now, and the history, your instructions, the tool definitions and any retrieved text all share the same desk. So how much desk is 64K?

That is a short novel, so a chat, an email or a translation sits comfortably. The squeeze comes from agents, because every step drops its tool output onto the desk and nothing leaves.

Real agents get around this by trimming or summarizing old steps. It works, but it is extra code to write and to debug. The cost side of the same loop is in Agent session cost: 50 steps on GLM, DeepSeek or MiniMax.

Sort your task: local or hosted#

The same question for a big open model is answered in Run DeepSeek locally or use an API. For a 4 GB model the answer depends on the task, so sort the task first and install second.

The hosted columns come from the CheapTokens model list. DeepSeek-V4-Flash has a 380K context, up to 32,768 output tokens per answer and one price at any hour. GLM-5.3-Flash has a 390K context and up to 8,192 output tokens. MiniMax-M2.7 has a 180K context, up to 8,192 output tokens, and suits agentic and coding work through an API.

One row the table can't capture: if the text must never leave your machine, that settles it in favor of local.

Bulk jobs: hours of GPU time or a small bill#

Say you tag 20,000 support tickets. Each one is about 600 tokens in and a 50-token label out.

Totals: 20,000 × 600 = 12,000,000 input tokens and 20,000 × 50 = 1,000,000 output tokens.

Generating those 1,000,000 tokens at NVIDIA's measured speeds takes 1,000,000 ÷ 133.4 = 7,496 s (about 2.1 hours) on the RTX 5070 and 1,000,000 ÷ 15.5 = 64,516 s (about 17.9 hours) on the Jetson Orin Nano. That counts generation only: reading the 12 million input tokens comes on top, and so does the electricity the card burns the whole time.

The hosted side of the same job, at current prices:

The table prices the job on each model, with an OpenRouter column. In total, with input and output counted together, each of them comes out cheaper than OpenRouter. On DeepSeek-V4-Flash, for instance, the 1,000,000 output tokens are billed at $0.10 per 1M.

Your job is probably not 20,000 tickets. Put your own numbers in:

Now you have both sides in the same units: hours of a machine you own against dollars on a live table.

Long documents and long answers#

Take a contract bundle of 120,000 tokens. It is bigger than 64,000, so a local Bolt E4B would need the bundle cut into pieces, asked piece by piece, and the answers stitched together. On a 380K window it goes in as one request: with a 2,000-token answer that is $0.002 at current prices. Which model fits a 100K-token document goes through the context and output limits model by model.

Long answers are the other limit. DeepSeek-V4-Flash can write up to 32,768 tokens in one answer, and at the same rough ¾ word per token (an assumption) 32,768 × 0.75 = 24,576 words, which is a long report or a whole code file. NVIDIA's page gives no maximum for Bolt E4B, so before you plan a 10,000-token output locally, test what your runtime actually returns.

Route by size: the same code, two destinations#

You don't have to pick one forever. Keep the small model for the small jobs and send the big ones to a hosted model. If your tool already talks to an OpenAI-compatible endpoint, the switch is a base URL, a key and a full model id.

import os
from openai import OpenAI

hosted = OpenAI(
    base_url='https://api.cheaptokens.dev/v1',
    api_key=os.environ['CHEAPTOKENS_API_KEY'],
)

def too_big_for_local(text: str, limit: int = 60_000) -> bool:
    return len(text) // 4 > limit  # rough: about 4 characters per token in English

document = open('contract.txt', encoding='utf-8').read()
if too_big_for_local(document):
    resp = hosted.chat.completions.create(
        model='deepseek-ai/DeepSeek-V4-Flash-0731',
        messages=[{'role': 'user', 'content': 'List the termination clauses:\n\n' + document}],
        max_tokens=2000,
    )
    print(resp.choices[0].message.content)
    print(resp.usage)

The 60,000 limit leaves room under the 64K window for instructions and the answer. When the call works you see the answer text, then a usage object with prompt_tokens and completion_tokens: multiply those by the prices on the pricing page to get what the request cost. The first-request steps are in the docs.

Start with the row of the table that matches your job. If it says local, download the GGUF and measure your own tokens per second before you plan around anyone's numbers.