A July 2025 research result put a hard number on a question agent builders keep dodging: how much of a tool list can you strip out per turn before accuracy breaks? The answer, for capable models, is almost all of it.
The 90% Tool-Pruning Result, Explained
Aggressive tool pruning removes most of an agent's available tools from any single turn's context while task accuracy holds — provided the model is strong enough to reason about what it dropped. In MemTool (arXiv 2507.21428), reasoning LLMs reached 90–94% tool-removal efficiency over a three-window average, while medium-sized models ranged from 0–60% . That spread is not a tuning gap you close with better prompts; it is a capability cliff that decides whether aggressive pruning is safe to attempt at all.
The reason pruning helps rather than hurts comes from how attention works. Anthropic frames the failure mode as "context rot": every token attends to every other token, so the relationships scale as n², and a bloated tool list taxes a finite attention budget even when each individual schema is small . Bigger windows did not fix this — the naive "include everything" strategy measurably degrades performance.
"If even a human engineer cannot confidently choose a tool," the agent does worse — bloated, overlapping toolsets confuse selection. — Anthropic, Effective context engineering for AI agents (source: Anthropic, 2025-09)
Two independent signals point the same way. LangChain reports that running RAG over tool descriptions improved tool-selection precision roughly 3× . And OpenAI's Agents SDK enforces the logic structurally: keep namespaces under 10 functions and defer the rest until runtime . The reframe: in capable models, the risk of over-loading the inference payload is larger than the risk of under-loading available tools. The seed practitioner discussion reflects this live debate.
What Context Engineering Actually Means — the Discipline Behind Tool Pruning
Context engineering is the discipline of curating exactly which tokens an LLM sees at each inference step — and tool pruning is one tactic inside it. Andrej Karpathy popularized the term on June 25, 2025, defining it as "the delicate art and science of filling the context window with just the right information for the next step" . The object being optimized is not a single call's wording but the stateful pipeline that curates, prunes, and orders every token across hundreds of turns.
That reframes context as an execution surface — system instructions, tool schemas, retrieved facts, intermediate logs, and handoff summaries visible to each call — rather than a passive prompt appendix. Cognition, the team behind Devin, puts the stakes plainly:
"Context engineering is effectively the #1 job of engineers building AI agents." — Cognition (source: Don't Build Multi-Agents)
The reason pruning matters is that the naive "include everything" strategy measurably hurts task completion. Anthropic's September 29, 2025 guide notes that 1M+ token windows did not solve "context rot": recall still degrades as token count rises because attention is finite and every token attends to every other . Anthropic also separates two sub-goals that must stay in order — maximize recall first through compaction and structured note-taking, then trim for precision. Conflating them causes compaction to discard facts the agent still needs mid-run.
Applying LangChain's Context Taxonomy to Tool Pruning
LangChain's four-operation model — write, select, compress, isolate — gives tool pruning a vocabulary, and each operation maps to a specific runtime decision . Per-turn tool removal like MemTool is the select lever applied to schemas, but it only works reliably when the other three are in place. Treat them as one system, not four independent tricks.
Write persists state outside the active window before it fills. Anthropic's multi-agent researcher saves its plan to an external memory store precisely because context exceeding ~200,000 tokens would be truncated mid-run . For long-horizon agents this is not optional — if a pruned tool is needed again three turns later, its calling contract must survive somewhere the model can re-read.
Select loads only the schemas the current subtask needs. OpenAI's hosted tool search defers large tool surfaces until runtime and recommends segmenting by namespace, ideally under 10 functions per namespace . That is the architectural mechanism MemTool operationalizes at the model level, where reasoning LLMs reached 90–94% tool-removal efficiency over a three-window average .
Compress reclaims budget without losing continuity. Claude Code auto-triggers compaction after roughly 95% of the window is consumed, and the spec is "maximize recall first, then trim for precision" — not oldest-first truncation . The ordering matters: dropping the oldest turns blindly can discard a tool result the agent still depends on.
Isolate fences scope across sub-agents. In Anthropic's research eval each sub-agent returned only a 1,000–2,000 token condensed summary to the orchestrator, and that system beat single-agent Opus 4 by 90.2% on an internal benchmark . Sharing full traces instead invites misaligned decisions — Cognition's "Flappy Bird" case had one sub-agent build a Mario-style background while another built a mismatched bird .
| Lever | Triggering condition | Token-budget impact | Failure it prevents |
|---|---|---|---|
| Write | State needed after truncation (~200K limit) | Moves bytes out of the window | Losing plans/tool contracts mid-run |
| Select | Toolset larger than the current subtask | Loads only needed schemas per turn | Wrong-tool selection from bloat |
| Compress | ~95% of window consumed | Summarizes older turns in place | Recall loss from blind truncation |
| Isolate | Parallel or read-heavy subtasks | Returns 1–2K token summaries only | Conflicting cross-agent decisions |
Handoff and Memory: Where Aggressive Tool Pruning Misfires
Aggressive tool pruning breaks when it strips context a downstream agent still needs. Anthropic specifies that every sub-agent handoff carry four fields — an objective, an output format, source/tool guidance, and clear scope boundaries . Drop any one and the agent re-discovers what it should have inherited, burning the tokens pruning was supposed to save.
Memory is the second failure point. LangChain warns that even long-context agents become slower, costlier, and distracted by stale content, so writing state to an external store before the window fills is not optional for runs past roughly 200,000 tokens . Truncate before you write and you lose plans and tool contracts mid-run.
Parallelism is the third. Cognition's rule is that read-only sub-agents — search, summarization — are safe to run in parallel, but write operations must stay single-threaded; parallel writes produce conflicting implicit decisions that surface later as plausible-but-wrong trajectories .
Finally, schema ambiguity at the tool level compounds across scale. Anthropic's Building Effective Agents appendix notes that switching a file-edit tool to require absolute paths fixed a recurring SWE-bench failure . Small changes that remove ambiguity pay off across thousands of turns.
Implementing Tool Pruning Incrementally
Start with an audit, not a rewrite. Log every tool schema loaded per turn and apply Anthropic's heuristic: if a human engineer cannot confidently choose among them, the model cannot either . That overlapping, ambiguous set is your pruning candidate list. Next, set a compaction threshold — Claude Code's documented trigger fires at roughly 95% window fill — but benchmark recall on your own task type before moving it. Tier your expectations by base model: MemTool's 90–94% tool-removal efficiency was measured on reasoning-tier LLMs, while medium models ranged 0–60% , so mid-tier deployments should pair selective loading with explicit task-gating before committing to aggressive per-turn pruning. Finally, audit MCP tool descriptions: Invariant Labs disclosed tool-poisoning via hidden instructions on April 1, 2025 — every schema you expose enlarges that surface. Prune to shrink both the token bill and the attack surface.
Frequently asked questions
What exactly does MemTool prune — the tool call or the schema?
MemTool prunes tool schemas — the descriptions and parameter specifications that occupy the context window — not the underlying tool implementations. The functions themselves stay callable; the agent simply sees fewer schemas per inference turn. MemTool's selective loading exposes only the subset relevant to the current task state, then re-evaluates on the next turn. In the arXiv evaluation, reasoning LLMs reached 90–94% tool-removal efficiency over a three-window average while medium-sized models ranged 0–60% . That efficiency is measured against schemas removed from active context, not tools deleted from the registry.
Does context engineering replace prompt engineering?
No — they optimize different objects. Prompt engineering tunes instruction wording for a single call; context engineering designs the stateful pipeline that decides what gets written, selected, compressed, or isolated across hundreds of turns . Andrej Karpathy framed the discipline as filling the context window with "just the right information for the next step" in a June 2025 post . At agent scale, context engineering subsumes most of the reliability problem; prompt engineering handles the residual quality of individual instructions.
When should I isolate sub-agents versus share full context across them?
Isolate for read-only or bounded sub-tasks — search, summarization, code scanning — where a sub-agent can return a condensed 1,000–2,000 token summary to the orchestrator . Share fuller context when agents make write decisions that depend on each other's state. Cognition's principle is that shared full traces prevent conflicting implicit decisions, illustrated by its "Don't Build Multi-Agents" post . The rule of thumb: isolate reads, single-thread writes.
How does Anthropic's compaction differ from truncating old messages?
Compaction maximizes recall first — it summarizes the full conversation into a dense representation — then trims for precision. Plain truncation drops the oldest tokens with no regard for what the agent still needs mid-task. Claude Code triggers auto-compaction after roughly 95% of the window is consumed . The ordering — recall before precision — is what preserves mid-task state that naive truncation silently discards.
What token overhead should I expect when moving to multi-agent with pruning?
Anthropic's measured figures: agents use roughly 4× more tokens than chat, and multi-agent systems use about 15× . In the same study, token usage alone explained 80% of BrowseComp performance variance, while token usage, tool calls, and model choice together explained 95% . Pruning reduces per-call schema overhead, but orchestration adds its own cost — budget both sides explicitly rather than assuming the savings net out.
Enjoyed this article? Subscribe to get new stories by email whenever they're published.
