TLDR: every agent resends its full tool definitions on every turn, burning thousands of tokens for zero information. I measured whether this is necessary. On 9 models, 8 is a serving-stack bug.
Every agentic turn, virtually all clients resend every tool definition. A typical agent carries 20–100 tools at 100–400 tokens per JSON schema. That is 2k–40k tokens of pure boilerplate, re-sent and re-billed on every single call. On a 200-turn trajectory, you pay for the same dictionary 200 times. The information content after turn one: zero. This is the industry’s most expensive copy-paste.
It should work
The chat-completions contract makes tools an optional
field per request, an optional field that some inference backends could
keep in the conversation state so that a client would send them once and
omit the field afterwards. The question is empirical. When you
omit tools on turn 2, does the model still call your
tool? I probed it.
Methodology
Two-turn conversation, single tool
package_tracker_lookup(tracking_number).
- Turn 1 (seed) sends the tool definitions normally; the model must
answer a parcel question with a structured
tool_calls. - Turn 2 (memory) replays the history — the assistant’s
tool_callsplus the matchingtool-role result — with a new parcel question andtoolsomitted. - The control leg resends everything, proving that any failure comes from the omitted field, not the model.
- The stub leg resends only
{"type":"function","function":{"name":"package_tracker_lookup"}}— name, no schema — to test how much of the definition the model still needs.
Results
tools omitted on turn 2, verdict per model:
| Model | Verdict |
|---|---|
| north-mini-code-free | ✅ structured tool_calls with correct
arguments ({"city": "Berlin"}), reproduced 3× — the only
one |
| deepseek-v4-flash-free | ❌ leaks
<｜DSML｜tool_calls><｜DSML｜invoke name="get_weather">...
into content (intent visible in
reasoning_content) |
| kimi-k2.6 | ❌ same DSML special-token soup in content |
| hy3-free | ❌ raw special-token soup in content |
| nemotron-3-ultra-free | ❌ raw special-token soup in content |
| mimo-v2.5-free | ❌ raw special-token soup in content |
| big-pickle | ❌ raw special-token soup in content |
| GLM (coding endpoint) | forgotten — “I don’t have access to a package tracking tool” |
| DeepSeek deepseek-flash | inline — emits internal
<｜DSML｜ invoke ...> markup in content
instead of a structured tool_calls |
The name-only (stub) leg is in the Workaround section below.
The one that works: north-mini-code-free
north-mini-code-free
(opencode Zen gateway, now retired) supported having the tool definition
sent only once. Verified three times: send tools on turn 1,
omit on turn 2 — clean, structured tool_calls still. Why do
the 8 others fail? Not the model. The server.
Bug reports
vLLM
The mechanism. vLLM’s per-model tool parsers (Hermes, Kimi K2,
DeepSeek V3/V3.1, …) set skip_special_tokens = False so
that tool-call framing tokens survive detokenization — but only
when request.tools is present. Omit
tools and the model still emits the tokens anyway (intent
comes from history), which decode as mojibake in content
(｜｜DSML｜tool_calls>...). North’s stack parses
unconditionally. That one gate is the entire difference between
“impossible” and “works.”
I proposed a fix upstream in vllm-project/vllm#48940:
a narrow elif that applies the same protection whenever the
history shows prior tool calling, changing nothing surfaced to the
client. With it, “send tools once, omit later” works
today on an unmodified OpenAI-compatible vLLM endpoint.
DeepSeek
I also filed deepseek-ai/DeepSeek-V3#1678:
the model has internalized the tool — it emits a perfectly formed
invocation — but with no tools field present, the decoding
layer has no channel to route it into, and the private DSML envelope
spills into user-visible content with
tool_calls: null.
Workaround: seed-then-name
Cannot drop tools entirely? Shrink it. The technique,
seed-then-name:
- Turn 1 — seed. Full definitions, exactly as today:
tools: [{type: "function", function: {name, description, parameters}}]. The one request that pays for the schema. - Turn 2+ — stub. Each tool by name only:
tools: [{type: "function", function: {name: "package_tracker_lookup"}}]— nodescription, noparameters. The field is present, so the tool-call parser and structured-output channel stay active (no DSML leakage); the payload is a few tokens per tool instead of hundreds.
Why it works: the model has already read the full schema in turn 1, and the history is re-read by the server on every request anyway. The semantics live in the history, not in this request’s schema.
Measured: DeepSeek reconstructs full, correct parameters from the
name alone ({"tracking_number": "TRK-9902-C"}). GLM answers
{} — detectable in the response; the fix is to resend the
full schema for one turn and retry.
Tools and refreshers
Agent practitioners know refreshing. When an important instruction drifts deep into the context and the model starts ignoring it, you restate it near the end. The text never left the window — old tokens just lose pull.
Today’s resend-everything behavior is a forced
refresher. The full tool dictionary is re-injected at the end
of the prompt every turn, at full price, whether you want it or not. And
it is all-or-nothing; you cannot keep read_file fresh and
let sql_query fade.
The two techniques above make refreshing opt-in:
- Send-once (north / the vLLM patch): zero refresh. Cheapest; on very long sessions definitions may go stale — your call when to accept that.
- Seed-then-name: a dial. The name-only stub is a minimal refresh for a few tokens; resend the full schema when you observe degradation (thin arguments, hallucinated parameter names). DeepSeek needed no full refresh at turn 2; GLM needed one immediately.
Tool definitions stop being a fixed tax and become a cache you manage: seed, keep warm cheaply, invalidate on observed staleness.