MCP spec should address tool schema token overhead (~1000 tokens/tool consumed per session) #2812
Replies: 5 comments 2 replies
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
-
|
A cross-implementation data point, since the numbers here so far are each from a single server (11 / 29 / 53 tools): I measured 13 real open-source MCP servers + agents (79 tools total) through one tokenizer so they're directly comparable. Full dataset + method is here; the parts relevant to the spec question: Per-server overhead (schema only, compact JSON):
That corroborates the ~100-tok floor and ~800–1,200-tok heavy tools measured upthread — it just generalizes across the ecosystem rather than one server. Two findings that I think sharpen @PengSpirit's "which tokens earn their place" and @gustavo-sec's "enforceable in CI": 1. A large slice of the bloat is tokens nobody authored. Serialization alone swings the bill ~20% on the identical tool — Fetch MCP is 236 tok compact vs 288 pretty-printed. Pydantic's 2. The single fattest tool in the set is Honesty caveat: my cross-server counts use Net: tiered / |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
MCP spec should address tool schema token overhead (~1000 tokens/tool consumed per session)
Date: 2026-05-28
Reporter: @luwei-will
Project: context-mode (Claude Code plugin)
Scope: 11 MCP tools — token footprint per tool definition
1. Executive Summary
MCP tool definitions consume 5–15× more tokens than the simplest possible schema for the same tool (type-only, no descriptions). In a typical Claude Code session with 20–30 registered MCP tools, the tool schema alone occupies 15–30 KB of context window before a single user message is sent. This is a structural inefficiency in the MCP protocol: context window capacity is consumed even when billing is amortized via prompt caching.
This issue documents measured token overhead of 11 production MCP tools, identifies the root cause, and proposes three protocol-level mitigations.
2. Measured Data
ctx_statsctx_batch_executectx_executectx_fetch_and_indexctx_execute_filectx_indexctx_searchctx_purgectx_insightctx_upgradectx_doctorctx_statsMethodology: Token counts measured via Anthropic's token counting API (
/v1/messages/count_tokens) with each tool definition serialized as its full JSON Schema including descriptions. Counts represent input tokens consumed per tool per session entry.Key observation: Heavy tools (
ctx_batch_execute,ctx_execute,ctx_fetch_and_index) each cost ~1,000 tokens to define. Light tools (ctx_stats,ctx_doctor) cost ~100 tokens. The 10× delta is entirely in JSON Schema size — parameter descriptions, type definitions, and nested object structures.3. Root Cause Analysis
3.1 Schema Bloat
MCP requires every tool to expose a full JSON Schema. For
ctx_batch_execute, the schema includes:commands: array of objects, each withlabel(string) andcommand(string)queries: array of stringsconcurrency: integer with range constraintstimeout: integerThe field descriptions (required by MCP for LLM reasoning) add ~400 tokens. The type definitions add ~300. The nested object structure for
commandsadds ~300 more. Total: ~1,000 tokens for one tool that a human could describe in two sentences.3.2 First-Turn Tax and Cache Fragility
Tool schemas are injected into the system prompt prefix. While prompt caching avoids re-billing cached prefixes on subsequent turns, the overhead is real in these scenarios:
With 20 tools averaging 500 tokens each, 10,000 tokens of context window are consumed by schemas regardless of billing amortization.
4. Impact on Real Workloads
From production usage (context-mode v1.0.151, 22 days, 2,600 conversations):
The billing cost is moderate. The reasoning capacity cost is the primary concern: a model reasoning over a complex task with 20 MCP tools registered has 10,000 fewer tokens available for actual work.
5. Proposed Mitigations
5.1 Tiered Schema Detail (Short-Term)
Allow MCP servers to register tools at two detail levels:
The model sees the discovery tier in the system prompt. When it decides to call a tool, the runtime injects the invocation-tier schema for that specific tool into the next turn.
{ "name": "ctx_batch_execute", "description": "Run commands in parallel, auto-index output, return matching sections.", "discovery_schema": { "type": "object", "properties": { "commands": {"type": "array"}, "queries": {"type": "array"}, "concurrency": {"type": "integer"}, "timeout": {"type": "integer"} } }, "invocation_schema": "<full schema with descriptions>" }Estimated saving: 60–70% of per-session schema tokens.
Tradeoff: Slightly reduced tool selection accuracy for rarely-used tools with non-obvious names.
5.2 Explicit Schema Versioning (Medium-Term)
Add a
schema_versionfield to MCP tool registrations. The host runtime tracks versions and only re-sends schemas to the model when the version changes. This makes the caching contract explicit rather than relying on implicit prompt-prefix matching, and enables delta updates (add/remove a single tool without invalidating the entire prefix).{ "name": "ctx_batch_execute", "schema_version": "2.1.0", "schema": { ... } }Estimated saving: Prevents full prefix invalidation on incremental tool changes.
5.3 Tool Namespacing (Long-Term)
Allow tools to be grouped under namespaces with shared schema prefixes. Instead of 20 top-level tools, expose 3–5 namespaces with 4–6 tools each. The namespace schema (shared parameter patterns, shared description vocabulary) is sent once; individual tools inherit and override.
Estimated saving: 30–40% of schema tokens via deduplication of repeated patterns.
6. Reproduction
7. Request
The MCP protocol is well-designed for capability discovery, but the cost model assumes schemas are cheap. They are not. Every token of schema is a token of context window — and context window capacity directly bounds model reasoning quality and session length.
Requested additions to the MCP specification:
schema_versionfield in tool registration for explicit cache invalidationThis would benefit all MCP implementations and hosts, not just context-mode.
Data from production usage of context-mode v1.0.151 over 22 days, 2,600 conversations.
Beta Was this translation helpful? Give feedback.
All reactions