What is AI agent tool calling and how does it work in 2026?

AI agent tool calling (also called function calling) is the mechanism by which an LLM selects and invokes external tools — APIs, databases, code execution, web search — to accomplish tasks that require real-world actions or data. The tool calling protocol separates model reasoning from deterministic execution: the LLM decides what tool to call and what parameters to pass, but the actual execution happens outside the model. How it works: (1) Tool definitions are loaded into the LLM's context — each tool has a name, description, and parameter schema (JSON Schema). (2) The user sends a request. (3) The LLM reasons about the request and decides whether a tool call is needed. (4) If yes, the LLM outputs a structured tool call: tool name + parameters (in JSON format). (5) The agent framework executes the tool call — calling the actual API, database, or function. (6) The result is injected back into the LLM's context. (7) The LLM reasons about the result and either calls another tool, provides a final answer, or asks for clarification. Key 2026 statistics: (1) 6+ sequential tool calls produce a failure rate of more than 50% — chaining is hard for agents (AAIF 2026). (2) Using more than ~20 tools in a single agent context degrades selection accuracy (AAIF 2026). (3) If tool enumeration pushes past 40, the agent is probably doing too much and should be two agents (AAIF 2026). (4) Description quality has the largest effect on selection accuracy — larger than tool name and larger than context. Optimizing descriptions alone produced roughly a 10x reduction in errors across 20,000+ tool evaluations (AAIF/Arcade 2026). (5) Tool definitions load into the LLM's context on every call, whether the tool is used or not. Multiple connected MCP servers can consume significant context before the user has asked a single question (AWS 2026). (6) Anthropic reports up to 85% token reduction by loading tool definitions only when relevant (lazy loading) (Anthropic 2026). (7) Anthropic's code execution pattern demonstrates a 98.7% token reduction by having agents write orchestration scripts instead of making individual tool calls (chatforest 2026). The tool calling protocol: (1) Initialize — client-server handshake (JSON-RPC initialize). (2) Tool discovery — get list of tools server supports (tools/list). (3) Tool call — call a specific tool and get results (tools/call). (4) Resource read — read resources exposed by server (resources/read). (5) Prompt get — get predefined prompt templates (prompts/get). (6) Notification — one-way notification, no response needed. (7) Cancel — cancel an in-progress request. MCP (Model Context Protocol) in 2026: MCP has become the standard protocol for connecting AI agents to tools. An MCP gateway sits between the agent and tool providers, handling discovery, authentication, routing, error normalization, and billing. One gateway can connect to 87+ tools through 47+ adapters. The agent never needs a hardcoded tool list — when new tools are added to the registry, every agent that queries the catalog picks them up automatically. Tool calling benchmarks: Anthropic 8.4, OpenAI 6.3 on tool calling accuracy. Google models perform well despite product complexity overhead. Open-source: Llama 3.1/3.2 has native tool calling support, production-ready in 2025-2026 (zylos 2026). The key: 'The intelligence is in the LLM. The hard engineering is in the tool infrastructure.' Tool calling is conceptually simple — the LLM picks a tool and passes parameters. The hard part is tool design, discovery, routing, error handling, orchestration, and security at production scale (AWS 2026, AAIF 2026, toolroute 2026, zylos 2026).

How do I design tools for AI agents?

Designing tools for AI agents is fundamentally different from designing APIs for programmers. Agents are not programmers — they don't reliably sequence calls, hold state, or know which function to call first. The tool abstraction problem: most MCP tools are designed for programmers. APIs expose capabilities (getUserId, getCalendarEvents) that a programmer sequences correctly. An agent doesn't do this reliably. Multiple benchmarks show that 6+ sequential tool calls produce a failure rate of more than 50%. The solution: build tools around tasks, not capabilities. A task like 'find the customer who complained last week and schedule a follow up' might touch five API endpoints. As five separate tools, that's a chaining problem with odds worse than a coin flip. As one tool oriented around that specific task, selection accuracy goes up and the deterministic parts stay deterministic. Tool design principles: (1) Build tools around tasks, not capabilities — 'Track this order and return a report' is a task. 'Get order ID' is a capability. Tools named and scoped to match how agents decompose work get selected more reliably. The Redis and Block/Square papers both show measurable accuracy gains from this approach. (2) Description quality is the 10x lever — Arcade's eval data shows that across 20,000+ tool evaluations, description quality has the largest effect on selection accuracy. Larger than tool name, larger than context. Tool schemas land near the bottom of the context window — that position means the description is the most recently processed information before the model makes its selection. It's functionally the last thing the model thinks about before choosing. Optimizing descriptions alone produced roughly a 10x reduction in errors. (3) Keep descriptions under 600 words — start with an action verb, write a short task-intent-formed sentence that tells the model what the tool does and when to reach for it. (4) Minimize context bloat — tool definitions load into the LLM's context on every call, whether the tool is used or not. Multiple MCP servers can consume significant context before the user asks a question. Use lazy loading: remove complex tool descriptions from always-loaded context and provide a separate tool for on-demand retrieval. Anthropic reports up to 85% token reduction by loading tool definitions only when relevant. (5) Return only what's needed — a tool that returns 50 fields per result fills context quickly. If 5 fields are sufficient for a decision, default the response to those 5 and provide a separate option to request a detailed view. Shifting to on-demand output cuts response tokens by roughly two-thirds (Anthropic). (6) Write helpful error messages — when a tool call fails, helpful errors steer the next attempt. 'Search requires 2 or more terms in query' tells the LLM exactly what to change. 'No results' gives it nothing, causing the model to give up or keep guessing. (7) Split multi-purpose tools — splitting a multi-purpose tool into several specific tools provides clarity and gives more granular results. (8) Use lazy loading discovery tools — remove complex tool descriptions from always-loaded context. Provide a separate tool for on-demand retrieval. The LLM only retrieves context when the task requires it. (9) Keep tool count under 20 — using more than ~20 tools in a single agent context degrades selection. If enumeration pushes past 40, the agent is probably doing too much and should be two agents. (10) Use an introspection tool — add a tool that directly calls an external LLM to provide targeted instructions for using other tools. The client LLM still makes the final call, but the introspection LLM interprets the need and provides correct values. A smaller, faster model handles this well and keeps costs reasonable. (11) Avoid semantic similarity between tools — semantic similarity between tools, too many options, and ambiguous naming contribute to confusion. Give each tool a distinct name and description. (12) Validate parameters — validate tool arguments against schemas, not just types. Add policy controls to every tool invocation. (13) Normalize errors — different tools return errors differently. Without normalization, every tool needs its own error parser. Use a gateway that normalizes errors into a consistent format. (14) Consider code execution orchestration — Anthropic's code execution pattern has agents write Python scripts that orchestrate entire workflows, running in a sandboxed Code Execution tool. This demonstrates a 98.7% token reduction vs individual tool calls. The key: 'Agents are not programmers. Building tools around tasks and intents, then obsessing over description quality, produces dramatically better selection accuracy and fewer failures at runtime.' Tool design is the highest-leverage optimization for agent tool use. Get the design right and everything else gets easier (AAIF 2026, AWS 2026, Anthropic 2026).

What is MCP and how does it work for AI agent tools?

MCP (Model Context Protocol) is the standard protocol for connecting AI agents to external tools, data sources, and services. It was introduced by Anthropic in 2024 and has become the dominant tool connectivity standard in 2026. What MCP provides: (1) Standardized protocol — MCP defines a standard way for agents to discover, call, and receive results from tools. Instead of each tool having its own API format, MCP provides a uniform interface. (2) Tool discovery — agents can dynamically discover what tools are available via tools/list. No hardcoded tool lists needed. When new tools are added to an MCP server, every connected agent picks them up automatically. (3) Tool calling — agents call tools via tools/call with structured parameters. Results come back in a standardized format. (4) Resource reading — agents can read resources (files, data) exposed by MCP servers via resources/read. (5) Prompt templates — MCP servers can expose predefined prompt templates via prompts/get. (6) Transport options — MCP supports stdio (local process communication) and SSE (Server-Sent Events for remote connections). Production environments use SSE for network-based connections to remote MCP servers. The MCP gateway pattern: an MCP gateway sits between the agent and the tool providers. It handles: (1) Discovery — query the catalog at runtime to learn what tools exist, what they do, and what inputs they accept. (2) Authentication — one key authenticates against all tools through adapters. The gateway holds upstream credentials. Supports BYOK (bring your own key) for enterprise plans. (3) Routing — different tools speak different protocols (REST, GraphQL, MCP, gRPC). The gateway provides a uniform way to call any of them. One endpoint (POST /api/v1/execute), same auth header, same response shape. (4) Error normalization — different tools return errors differently. The gateway normalizes all errors into a consistent format. (5) Billing and spend controls — autonomous agents need guardrails. Each paid tool call deducts from a balance. When credits hit zero, calls stop. No surprise bills. MCP ecosystem in 2026: (1) mcp-agent (8.2K GitHub stars) — provides Temporal-backed durable execution. Every workflow step is a Temporal Activity with automatic retry, timeout handling, and full execution history. If the process crashes, Temporal replays history and resumes from the last completed step. (2) Mastra (22.3K stars, $13M funding) — TypeScript graph engine with suspend/resume. (3) fast-agent — first framework with full MCP Sampling support (servers requesting LLM completions) and Elicitation support (servers requesting user input). Implements the MAKER error reduction pattern. (4) OpenAI Agents SDK — built-in MCP server tool calling. Agent's mcp_servers property auto-aggregates tools from all connected servers. (5) Claude Agent SDK — five-layer stack: MCP (connectivity) to Skills to Agent to Subagents to Agent Teams. Uses programmatic tool orchestration — Claude writes Python scripts that orchestrate workflows in a sandboxed Code Execution tool. MCP specification additions (2026): (1) Async Tasks (SEP-1686) — for long-running tool operations. (2) Enhanced Sampling (SEP-1577) — for server-side agent loops. (3) The 'inverted agent' pattern — where the MCP server controls the workflow while the client provides intelligence. When to use MCP vs API vs function calling: (1) MCP — when you need standardized tool discovery, multiple tool providers, dynamic tool registration, or cross-framework interoperability. (2) Direct API — when you have a single tool, simple integration, or need maximum control. (3) Function calling — when you're using a single LLM provider's native function calling format. In practice, a single agent might use all three: shell out to a CLI, call two MCP servers, run internal validation functions, and use the LLM provider's native function calling. The key: 'MCP has become the standard protocol for connecting AI agents to tools.' It solves the tool management problem — discovery, auth, routing, error normalization, billing — so the agent code stays clean. Use an MCP gateway for production agents with multiple tools (toolroute 2026, chatforest 2026, AWS 2026).

How do I orchestrate multiple tools in an AI agent workflow?

Orchestrating multiple tools in an AI agent workflow is the difference between a demo and a production agent. A single tool call is simple: request, response, done. But real workflows involve chains of tool calls across multiple servers, conditional branching, parallel execution, human approval gates, and error recovery. Without orchestration, you rely on the LLM to manage all of this in its context window — which works for simple cases but fails in predictable ways. Orchestration patterns: (1) Sequential — tools execute in order, each receiving prior output. Use case: data pipelines, ETL workflows. Example: search web, analyze results, generate report, send email. Each step depends on the previous step's output. (2) Parallel — multiple tool calls execute concurrently. Use case: multi-source search, fan-out aggregation. Example: search 3 databases simultaneously, merge results. Faster than sequential when there are no dependencies. (3) Router — LLM-based routing to specialized sub-workflows. Use case: task triage, intent classification. Example: classify user intent (refund vs technical vs sales), route to the appropriate tool workflow. (4) Evaluator-Optimizer — generate, evaluate, refine loop. Use case: content generation, code review. Example: generate content, evaluate quality, refine if needed, repeat until quality threshold met. (5) Orchestrator-Workers — central agent delegates to specialists. Use case: complex multi-domain tasks. Example: orchestrator agent delegates research to research agent, writing to writer agent, review to editor agent. (6) Map-Reduce — map inputs to parallel tasks, reduce results. Use case: batch processing, data analysis. Example: map 100 documents to parallel summarization tasks, reduce into a single report. Orchestration frameworks: (1) mcp-agent (8.2K stars) — Temporal-backed durable execution. Every step is a Temporal Activity with automatic retry, timeout, and full execution history. If the process crashes, Temporal replays and resumes. No work is lost. (2) Mastra (22.3K stars) — TypeScript graph engine with suspend/resume. (3) fast-agent — first framework with full MCP Sampling and Elicitation support. Implements MAKER error reduction pattern. (4) LangGraph — graph-based orchestration with checkpointing, HITL, and time-travel debugging. (5) OpenAI Agents SDK — built-in MCP support, tracing, and guardrails. (6) Claude Agent SDK — programmatic tool orchestration via code execution. Code execution orchestration (Anthropic's pattern): instead of making individual tool calls (which each consume context for the tool definition, call, and result), the agent writes a Python script that orchestrates the entire workflow. The script runs in a sandboxed Code Execution tool. This demonstrates a 98.7% token reduction vs individual tool calls. The script handles sequencing, error handling, and data passing — all outside the LLM's context window. Error handling and retry: (1) 3-tier fallback chain — resolve directly when confidence is high, retry with reformulated query when it isn't, escalate to explicit clarification request rather than forcing a tool call when even the retry fails. (2) Automatic retry with exponential backoff — for transient failures (network timeouts, rate limits). (3) Circuit breakers — if a tool fails repeatedly, stop calling it and fall back to an alternative. (4) Graceful degradation — if a tool is unavailable, provide a partial result or alternative path. (5) Error normalization — different tools return errors differently. Normalize all errors into a consistent format so the agent can handle them uniformly. (6) Helpful error messages — 'Search requires 2 or more terms in query' tells the LLM what to change. 'No results' gives it nothing. (7) Timeout handling — set timeouts on all tool calls. If a tool doesn't respond in time, cancel and retry or fall back. (8) Dead letter queues — failed tool calls go to a dead letter queue for investigation, not automatic retry. Checkpointing and durability: (1) Checkpoint progress — so a crashed agent can resume. mcp-agent uses Temporal for this. LangGraph uses checkpointers. (2) State propagation — pass intermediate results between steps. (3) Idempotency — ensure tool calls are idempotent so retries don't cause duplicate effects. (4) Transaction management — for multi-step workflows that need atomicity, implement rollback or compensation. Token optimization: (1) Lazy tool loading — load tool definitions only when relevant. 85% token reduction (Anthropic). (2) Code execution orchestration — write scripts instead of individual calls. 98.7% token reduction (Anthropic). (3) On-demand output — return only needed fields, not all 50. Two-thirds token reduction (Anthropic). (4) Tool result compression — compress large tool results before injecting into context. The key: 'Production workflows demand more than ad-hoc tool calls. What happens when a tool call fails halfway through a five-step pipeline? How do you checkpoint progress so a crashed agent can resume? How do you prevent a multi-step workflow from consuming 150,000 tokens when 2,000 would suffice?' Orchestration frameworks solve these problems by managing workflow state externally — outside the LLM's context window. The LLM still makes decisions, but the framework handles sequencing, retries, checkpointing, and state propagation (chatforest 2026, toolsku 2026, machinelearningmastery 2026).

How do I handle tool errors in AI agents?

Handling tool errors in AI agents is critical because tool calls fail frequently in production — network timeouts, rate limits, invalid parameters, service unavailability, and unexpected response formats are all common. Without proper error handling, a single tool failure can derail the entire agent workflow. Common tool errors: (1) Network errors — timeouts, connection refused, DNS resolution failure. (2) Rate limiting — 429 Too Many Requests. (3) Authentication errors — 401 Unauthorized, 403 Forbidden (expired or invalid API key). (4) Parameter errors — 400 Bad Request (invalid or missing parameters). (5) Service unavailable — 503 Service Unavailable (tool provider is down). (6) Unexpected response format — tool returns data in a different format than expected. (7) Partial results — tool returns some data but not all. (8) Semantic errors — tool executes successfully but returns wrong or irrelevant results. Error handling strategies: (1) 3-tier fallback chain — (a) Resolve directly when confidence is high: the tool call succeeds and returns valid results. (b) Retry with reformulated query when confidence is low: the tool call fails or returns poor results, reformulate the query and retry. (c) Escalate to explicit clarification request: when even the retry fails, ask the user for clarification rather than forcing a tool call. This prevents infinite retry loops. (2) Automatic retry with exponential backoff — for transient failures (network timeouts, rate limits), retry automatically with increasing delays: 1s, 2s, 4s, 8s. Set a max retry count (3-5) to prevent infinite loops. (3) Circuit breakers — if a tool fails repeatedly (e.g., 5 consecutive failures), stop calling it for a cooldown period (e.g., 60 seconds). This prevents cascading failures and gives the tool provider time to recover. After the cooldown, try again. If it still fails, fall back to an alternative. (4) Graceful degradation — if a tool is unavailable, provide a partial result or alternative path. Example: if the web search tool is down, try the cached results or a different search provider. If no alternative exists, inform the user that the tool is temporarily unavailable. (5) Error normalization — different tools return errors differently. Tavily returns errors differently than Resend, which returns errors differently than Stripe. Without normalization, every tool needs its own error parser. Use a gateway that normalizes all errors into a consistent format: {error_type, message, retryable, suggested_action}. (6) Helpful error messages — when a tool call fails, the error message should tell the LLM exactly what to change. 'Search requires 2 or more terms in query' tells the LLM to add more terms. 'API key expired' tells the LLM to use a different tool. 'No results' gives the LLM nothing — it will either give up or keep guessing. Always include actionable information in error messages. (7) Timeout handling — set timeouts on all tool calls (e.g., 30 seconds for API calls, 60 seconds for complex operations). If a tool doesn't respond in time, cancel the request and either retry or fall back. Use async/await with timeout for non-blocking timeout handling. (8) Dead letter queues — failed tool calls go to a dead letter queue for investigation, not automatic retry. This prevents retry loops and allows post-hoc analysis of failures. (9) Parameter validation before calling — validate parameters against the tool's schema before making the call. Catch parameter errors before they reach the tool provider. This saves a network round-trip and provides immediate feedback to the LLM. (10) Response validation after calling — validate the tool's response against the expected schema. If the response doesn't match, treat it as an error and retry or fall back. (11) Idempotency — ensure tool calls are idempotent so retries don't cause duplicate effects. If a tool call is not idempotent (e.g., sending an email), use idempotency keys to prevent duplicates. (12) Logging and monitoring — log all tool calls, errors, and retries. Monitor error rates per tool. Set up alerts when error rates exceed thresholds. (13) Fallback tools — if a primary tool fails, fall back to an alternative tool that provides similar functionality. Example: if the primary search API is down, fall back to a secondary search API. (14) Human escalation — for critical tool failures that can't be automatically resolved, escalate to a human. Example: if a payment tool fails, don't retry automatically — escalate to a human to handle the payment manually. (15) MAKER error reduction pattern — fast-agent implements this structured approach to reducing errors in multi-step agent workflows through validation and correction loops. Each step's output is validated, and if validation fails, a correction loop is triggered. The key: 'When a tool call fails, helpful errors can steer the next attempt. A response that says search requires 2 or more terms in query tells the LLM exactly what to change. A response that returns only no results gives it nothing, causing the model to either give up or keep guessing.' Error handling is not an afterthought — it's a core part of tool infrastructure. Design your error handling strategy before deploying to production (AWS 2026, machinelearningmastery 2026, chatforest 2026, toolsku 2026).

July 13, 2026

TL;DR — AI agent multiple tools in 2026: tool calling, MCP, and orchestration. 6+ sequential tool calls = 50%+ failure rate. 20+ tools in context degrades selection. Description quality = 10x error reduction. Anthropic: 85% token reduction via lazy tool loading. 98.7% token reduction via code execution orchestration. MCP: 87 tools, 47 adapters, one endpoint. Tool design: tasks not capabilities. Orchestration patterns: sequential, parallel, router, evaluator-optimizer, orchestrator-workers, map-reduce. Frameworks: mcp-agent (8.2K stars, Temporal), Mastra (22.3K stars), fast-agent (MCP Sampling + Elicitation). Tool security: per-tool scoping, schema validation, policy controls. Error handling: 3-tier fallback chain.

AI Agent Multiple Tools in 2026: Tool Calling, MCP, Orchestration, and Production Patterns

The intelligence is in the LLM. The hard engineering is in the tool infrastructure. When an agent needs to search the web, send emails, query databases, generate images, and scan code — each tool has its own API key, authentication scheme, rate limits, error format, and SDK. You are no longer building an agent — you are building a tool management platform.

Key Statistics

Metric	Value	Source
Failure rate (6+ sequential tool calls)	50%+	AAIF 2026
Tool count that degrades selection	20+	AAIF 2026
Tool count suggesting split into 2 agents	40+	AAIF 2026
Description quality error reduction	10x	AAIF/Arcade 2026
Tool evaluations measured	20,000+	AAIF/Arcade 2026
Token reduction (lazy tool loading)	85%	Anthropic 2026
Token reduction (code execution orchestration)	98.7%	Anthropic 2026
Token reduction (on-demand output)	~67%	Anthropic 2026
MCP tools via gateway	87+	toolroute 2026
MCP adapters	47+	toolroute 2026
mcp-agent GitHub stars	8,200	chatforest 2026
Mastra GitHub stars	22,300	chatforest 2026
Anthropic tool calling benchmark	8.4	zylos 2026
OpenAI tool calling benchmark	6.3	zylos 2026

Tool Design Principles

Principle	What It Means	Impact
Tasks not capabilities	Build tools around tasks, not API endpoints	50%+ failure rate eliminated for multi-step tasks
Description quality	Under 600 words, action verb, task-intent sentence	10x error reduction
Lazy loading	Load tool definitions only when relevant	85% token reduction
On-demand output	Return 5 fields by default, not 50	~67% token reduction
Helpful errors	'Search requires 2+ terms' not 'No results'	Steers next attempt correctly
Split multi-purpose tools	Specific tools provide clarity	Better selection accuracy
Keep under 20 tools	More than 20 degrades selection	If 40+, split into 2 agents
Distinct naming	Avoid semantic similarity between tools	Reduces confusion

Sources: AAIF (2026), AWS (2026), Anthropic (2026).

Orchestration Patterns

Pattern	How It Works	Use Case	Example
Sequential	Tools execute in order, each receives prior output	Data pipelines, ETL	Search → analyze → report → email
Parallel	Multiple tool calls execute concurrently	Multi-source search	Search 3 databases simultaneously
Router	LLM routes to specialized sub-workflows	Task triage	Classify intent → route to workflow
Evaluator-Optimizer	Generate, evaluate, refine loop	Content generation	Generate → evaluate → refine → repeat
Orchestrator-Workers	Central agent delegates to specialists	Multi-domain tasks	Orchestrator → research + write + review
Map-Reduce	Map inputs to parallel tasks, reduce results	Batch processing	100 docs → parallel summarize → merge

Source: chatforest (2026).

Orchestration Frameworks

Framework	Stars	Key Feature	Best For
mcp-agent	8.2K	Temporal-backed durable execution	Production workflows needing durability
Mastra	22.3K	TypeScript graph engine, suspend/resume	TypeScript teams
fast-agent	—	MCP Sampling + Elicitation, MAKER pattern	Inverted agent pattern
LangGraph	24K	Checkpointing, HITL, time-travel	Enterprise production
OpenAI Agents SDK	—	Built-in MCP, tracing, guardrails	OpenAI shops
Claude Agent SDK	—	Code execution orchestration (98.7% token reduction)	Anthropic-native

Sources: chatforest (2026), presenc.ai (2026).

Tool Orchestration Architecture

flowchart TD User["User Request"] --> LLM["LLM Reasoning\nDecides which tools to call\nBased on tool definitions in context"] LLM --> Discovery{"Tool Discovery\nDynamic registration via MCP\ntools/list → available tools\nLazy loading: only relevant tools"} Discovery --> Select{"Tool Selection\nDescription quality = 10x error reduction\n< 20 tools in context\nTasks not capabilities"} Select -->|Sequential| Seq["Sequential Pattern\nStep 1 → Step 2 → Step 3\nEach receives prior output\n6+ calls = 50%+ failure"] Select -->|Parallel| Par["Parallel Pattern\nMultiple calls concurrently\nNo dependencies\nFaster execution"] Select -->|Router| Rte["Router Pattern\nClassify intent\nRoute to sub-workflow"] Select -->|Code Exec| Code["Code Execution Pattern\nAgent writes Python script\n98.7% token reduction\nSandboxed execution"] Seq --> Gateway["MCP Gateway\nOne endpoint, any tool\n87+ tools, 47+ adapters\nAuth, routing, error normalization, billing"] Par --> Gateway Rte --> Gateway Code --> Sandbox["Sandboxed Code Execution\nNo network access\nNo production credentials\nStrict allowlist"] Gateway --> Execute["Execute Tool Call\nValidate params against schema\nPolicy controls on every call"] Sandbox --> Execute Execute --> Error{"Error Handling\n3-tier fallback:\n1. Resolve directly\n2. Retry with reformulated query\n3. Escalate to human\nCircuit breakers, timeout, dead letter queue"} Error -->|Success| Result["Tool Result\nOn-demand output (5 fields not 50)\n~67% token reduction"] Error -->|Retry| Gateway Error -->|Escalate| Human["Human Escalation"] Result --> Context["Inject into LLM Context\nCompressed result\nNext reasoning step"] Context --> LLM

Source: AAIF (2026), AWS (2026), chatforest (2026), toolroute (2026).

Token Optimization for Tool Use

Strategy	Token Reduction	How
Lazy tool loading	85%	Load definitions only when relevant
Code execution orchestration	98.7%	Agent writes scripts instead of individual calls
On-demand output	~67%	Return 5 fields by default, not 50
Tool result compression	40-60%	Compress large results before injecting
Introspection tool	Variable	Small LLM provides targeted instructions
Tool splitting	20-30%	Specific tools = smaller definitions
Context engineering	Variable	Structure prompts to minimize tool context

Source: Anthropic (2026), AWS (2026).

Implementation Guide

Phase	What to Do	Timeline
1. Inventory tools	List all tools the agent needs, their APIs, auth, rate limits	Week 1
2. Design tools around tasks	Not capabilities. One tool per task, not per API endpoint	Weeks 1-2
3. Write descriptions	Under 600 words, action verb, task-intent sentence	Weeks 1-2
4. Set up MCP gateway	One endpoint, auth, routing, error normalization, billing	Weeks 2-3
5. Implement tool discovery	Dynamic registration via tools/list, lazy loading	Weeks 2-3
6. Add parameter validation	Validate against schemas before calling	Weeks 3-4
7. Implement orchestration	Choose pattern: sequential, parallel, router, etc.	Weeks 3-5
8. Add error handling	3-tier fallback, retry, circuit breakers, dead letter queue	Weeks 4-5
9. Implement on-demand output	Return 5 fields by default, detailed view on request	Weeks 4-6
10. Add code execution	Sandboxed Python for complex orchestration (98.7% token reduction)	Weeks 5-7
11. Set up monitoring	Log all tool calls, errors, retries, latency, cost	Weeks 6-7
12. Add spend controls	Budget per tool, per workflow, per day	Weeks 6-8
13. Test tool selection	DeepTeam: test selection accuracy, parameter accuracy	Weeks 7-8
14. Security hardening	Per-tool scoping, schema validation, policy controls	Weeks 8-9
15. Production deployment	With monitoring, alerts, cost controls, error tracking	Weeks 9-10

Best Practices

Build tools around tasks, not capabilities — 'Track this order and return a report' is a task. 'Get order ID' is a capability. Tasks get selected more reliably. 6+ sequential capability calls = 50%+ failure (AAIF 2026).
Obsess over description quality — it's the 10x lever. Under 600 words, action verb, task-intent sentence. Description is the last thing the model thinks about before choosing. 10x error reduction from description optimization alone (AAIF 2026).
Keep tool count under 20 — more than 20 tools degrades selection. If you need 40+, split into two agents. Use lazy loading to keep context lean (AAIF 2026).
Use lazy loading — load tool definitions only when relevant. 85% token reduction. Use a discovery tool for on-demand retrieval (Anthropic 2026).
Return only what's needed — 5 fields by default, not 50. On-demand detailed view. ~67% token reduction (Anthropic 2026).
Write helpful error messages — 'Search requires 2+ terms' not 'No results'. Actionable errors steer the next attempt. Non-actionable errors cause giving up or guessing (AWS 2026).
Use an MCP gateway — one endpoint, any tool. Handles discovery, auth, routing, error normalization, billing. 87+ tools, 47+ adapters through one interface (toolroute 2026).
Consider code execution orchestration — for complex workflows, have the agent write a Python script that orchestrates tools. 98.7% token reduction. Sandboxed execution (Anthropic 2026).

FAQ

How many tools should an AI agent have?

The optimal number of tools for an AI agent depends on the agent's scope, but research in 2026 provides clear guidelines. The 20-tool limit: using more than ~20 tools in a single agent context degrades selection accuracy. This is because the LLM must process all tool definitions in its context window on every call, whether the tool is used or not. As the number of tools grows, the LLM's ability to reason degrades (context rot), and it makes poorer choices — calling the wrong tool, choosing incorrect parameters, or failing to select any tool. The 40-tool split: if tool enumeration pushes past 40, the agent is probably doing too much and should be two agents. This is a design signal that the agent's scope is too broad. Instead of one agent with 40 tools, split into two agents with 20 tools each, each focused on a specific domain. Why tool count matters: (1) Context bloat — tool definitions load into the LLM's context on every call. 20 tools with 200-word descriptions = 4,000 tokens of tool definitions per call. 40 tools = 8,000 tokens. This is before the user has asked a single question. (2) Confusion — as the number of tools grows, semantic similarity between tools increases. The LLM has to distinguish between similar tools, which leads to wrong selections. Ambiguous naming compounds this. (3) Reasoning degradation — as context fills with tool definitions, the LLM's ability to reason degrades. Research on context rot shows that LLMs process information less effectively as context grows. (4) Retry compounding — when the LLM selects the wrong tool, it retries. Retries add more context (the failed call and its error), further degrading performance. How to stay under 20 tools: (1) Build tools around tasks, not capabilities — instead of 5 API endpoints as 5 tools, create 1 tool that handles the entire task. This reduces tool count and improves selection accuracy. (2) Use lazy loading — don't load all tool definitions at once. Load a discovery tool that retrieves specific tool definitions on demand. Anthropic reports 85% token reduction with this approach. (3) Split into multiple agents — if you need 40+ tools, split into 2+ agents, each with under 20 tools. Use a router pattern to direct requests to the appropriate agent. (4) Use an introspection tool — instead of loading all tool definitions, provide a tool that queries an external LLM for targeted instructions on using other tools. This keeps context lean. (5) Merge similar tools — if two tools do similar things, merge them into one with optional parameters. (6) Remove unused tools — regularly audit which tools are actually being called. Remove tools that are never or rarely used. (7) Use progressive disclosure — start with a small set of core tools. As the conversation progresses and the agent's needs become clearer, dynamically add more tools. When to exceed 20 tools: (1) With lazy loading — if tools are loaded on demand, the LLM only sees the tools it needs for the current task. You can have 100+ tools in the registry but only 5-10 in context at any time. (2) With code execution orchestration — the agent writes a script that calls tools directly, without loading all definitions into context. 98.7% token reduction. (3) With a gateway — an MCP gateway handles tool discovery and routing. The agent queries the gateway for available tools, and the gateway returns only relevant ones. The key: 'Using more than ~20 tools in a single agent context degrades selection. If enumeration pushes past 40, the agent is probably doing too much and should be two agents.' Tool count is a design constraint, not a technical limitation. Design your agent with a focused scope, build tools around tasks, use lazy loading, and split into multiple agents when the scope grows (AAIF 2026, AWS 2026, Anthropic 2026).

What is the difference between MCP and function calling?

MCP (Model Context Protocol) and function calling are related but serve different purposes in AI agent tool use. Function calling: (1) What it is — a feature of LLM providers (OpenAI, Anthropic, Google) that allows the model to output structured tool calls (tool name + parameters in JSON) instead of free-text responses. (2) Scope — single LLM provider. Each provider has its own function calling format. (3) Tool definitions — hardcoded in the application. The developer defines tools, their schemas, and their implementations. (4) Discovery — static. Tools are defined at development time. Adding a new tool requires code changes. (5) Transport — the function call is output by the LLM and executed by the application. No standard transport protocol. (6) Authentication — managed by the application. Each tool's API key is managed separately. (7) Error handling — custom per tool. Each tool has its own error format and handling logic. (8) Best for — simple agents with a few tools, single LLM provider, prototyping. MCP (Model Context Protocol): (1) What it is — a standard protocol for connecting AI agents to tools, data sources, and services. Introduced by Anthropic in 2024, became the dominant standard in 2026. (2) Scope — cross-provider. MCP works with any LLM that supports tool calling. Not tied to a specific provider. (3) Tool definitions — dynamic. Tools are discovered at runtime via tools/list. Adding a new tool to an MCP server automatically makes it available to all connected agents. (4) Discovery — dynamic. Agents query MCP servers for available tools at runtime. No hardcoded tool lists. (5) Transport — standardized. MCP supports stdio (local) and SSE (remote). JSON-RPC 2.0 for message format. (6) Authentication — centralized via MCP gateway. One key authenticates against all tools. The gateway holds upstream credentials. Supports BYOK. (7) Error handling — normalized. The MCP gateway normalizes errors from different tools into a consistent format. (8) Best for — production agents with multiple tools, multi-provider setups, dynamic tool registration, cross-framework interoperability. Key differences: (1) Standardization — function calling is provider-specific. MCP is a cross-provider standard. (2) Discovery — function calling has static tool definitions. MCP has dynamic discovery. (3) Transport — function calling has no standard transport. MCP has stdio and SSE. (4) Authentication — function calling requires per-tool auth management. MCP centralizes auth via gateway. (5) Error handling — function calling requires per-tool error parsing. MCP normalizes errors. (6) Ecosystem — function calling is built into LLM providers. MCP has a growing ecosystem of servers, gateways, and frameworks (mcp-agent, Mastra, fast-agent). When to use each: (1) Function calling — when you have a simple agent with 3-5 tools, a single LLM provider, and don't need dynamic tool discovery. (2) MCP — when you have a production agent with 10+ tools, multiple tool providers, need dynamic tool registration, want cross-framework interoperability, or need centralized auth and error handling. (3) Both — in practice, a single agent might use both: function calling for the LLM's native tool calling format, and MCP for tool discovery and connectivity. The LLM uses function calling to make tool calls, but the tools are discovered and connected via MCP. The key: 'In practice, nobody picks just one pattern. A single agent might shell out to a CLI, call two MCP servers, run internal validation functions, and use the LLM provider's native function calling.' Function calling is how the LLM makes tool calls. MCP is how tools are discovered, connected, and managed. They work together, not in competition (jamwithai 2026, toolroute 2026, chatforest 2026).

How do I implement parallel tool calls in AI agents?

Parallel tool calls allow an AI agent to execute multiple tools simultaneously when there are no dependencies between them. This can significantly reduce latency for multi-tool workflows. When to use parallel tool calls: (1) Multi-source search — search 3 databases or APIs simultaneously and merge results. (2) Fan-out aggregation — call multiple tools to gather different types of information, then combine. (3) Independent operations — perform operations that don't depend on each other (e.g., send email + update database + log event). (4) Batch processing — process multiple items in parallel (map-reduce pattern). When NOT to use parallel tool calls: (1) Sequential dependencies — when each tool's input depends on the previous tool's output. Use sequential pattern instead. (2) Rate-limited APIs — if the tool provider has strict rate limits, parallel calls may hit the rate limit. (3) Shared resources — if tools access the same resource (e.g., same database row), parallel calls may cause conflicts. (4) Cost-sensitive workflows — parallel calls consume more tokens simultaneously. If cost is a concern, sequential may be cheaper. How to implement parallel tool calls: (1) LLM decides parallel calls — modern LLMs (GPT-5, Claude 4.5, Gemini 3) can output multiple tool calls in a single response when they determine the calls are independent. The LLM outputs an array of tool calls, and the framework executes them in parallel. (2) Framework parallel execution — the agent framework executes all tool calls in the array simultaneously using async/await or threading. Results are collected as they complete. (3) Result collection — once all parallel calls complete, results are collected and injected into the LLM's context for the next reasoning step. (4) Error handling — if one parallel call fails, the framework can either: (a) Return partial results (successful calls' results + error for failed call). (b) Retry the failed call. (c) Fail the entire batch. Choose based on your use case. Implementation with MCP: (1) Use async/await for non-blocking parallel execution. (2) Use asyncio.gather() to run multiple tool calls concurrently. (3) Use asyncio.wait_for() with timeout for each call. (4) Use try/except per call to handle individual failures. (5) Collect results and handle partial failures. Implementation with LangGraph: (1) Use Send() API to fan out to multiple nodes in parallel. (2) Each node executes a tool call independently. (3) Reducer function merges results from all parallel nodes. (4) Conditional edges determine next steps based on combined results. Implementation with OpenAI Agents SDK: (1) The SDK supports parallel tool calls natively. (2) When the LLM outputs multiple tool calls, the SDK executes them in parallel. (3) Results are collected and passed back to the LLM. Best practices: (1) Set per-call timeouts — don't let one slow call block the entire batch. Use asyncio.wait_for() with a 30-second timeout per call. (2) Handle partial failures — decide whether to return partial results, retry failed calls, or fail the batch. Document this behavior clearly. (3) Limit concurrency — don't launch 100 parallel calls. Use a semaphore to limit concurrency (e.g., 10 concurrent calls). This prevents overwhelming the tool provider. (4) Monitor latency — track per-call latency and total batch latency. Identify slow tools that bottleneck parallel execution. (5) Use connection pooling — for HTTP-based tools, use connection pooling to reduce connection overhead for parallel calls. (6) Consider rate limits — check the tool provider's rate limits before launching parallel calls. Add rate limiting to your framework if needed. (7) Log all parallel calls — log each call's start time, end time, status, and result. This helps debug parallel execution issues. The key: parallel tool calls can significantly reduce latency when tools are independent, but they add complexity in error handling, rate limiting, and result collection. Use them when the latency benefit outweighs the complexity (toolsku 2026, chatforest 2026, machinelearningmastery 2026).

What is code execution orchestration for AI agent tools?

Code execution orchestration is Anthropic's pattern for reducing token consumption in multi-tool workflows by having the agent write Python scripts that orchestrate tools, instead of making individual tool calls. It demonstrates a 98.7% token reduction compared to individual tool calls. The problem it solves: in a traditional tool calling workflow, each tool call consumes context for: (1) the tool definition (name, description, schema), (2) the tool call (tool name + parameters), (3) the tool result (output data). For a 10-step workflow with 10 tools, that's 10 tool definitions + 10 tool calls + 10 tool results = 30 context injections. Each injection adds hundreds to thousands of tokens. A 10-step workflow can easily consume 50,000+ tokens just for tool calling overhead. How code execution orchestration works: (1) The agent writes a Python script that orchestrates the entire workflow. The script calls tools directly (via function calls, API calls, or MCP), handles sequencing, error handling, and data passing — all in code. (2) The script runs in a sandboxed Code Execution tool — no network access (or restricted access), no production credentials, minimal filesystem permissions. (3) The script's output (the final result) is the only thing injected back into the LLM's context. Not the intermediate tool calls, not the intermediate results — just the final output. (4) The LLM reasons about the final output and either provides a response or writes another script for the next step. Token savings: (1) Traditional: 10 tool definitions (10K tokens) + 10 tool calls (5K tokens) + 10 tool results (35K tokens) = 50K tokens. (2) Code execution: 1 tool definition (Code Execution, 200 tokens) + 1 tool call (the script, 2K tokens) + 1 tool result (the final output, 1K tokens) = 3.2K tokens. (3) Savings: 50K - 3.2K = 46.8K tokens. 98.7% reduction. When to use code execution orchestration: (1) Multi-step workflows — 5+ tool calls in sequence. The more steps, the greater the savings. (2) Complex data processing — when the workflow involves data transformation, filtering, aggregation, or analysis between tool calls. (3) Repetitive tool calls — when the agent calls the same tool multiple times with different parameters (e.g., searching 10 different queries). (4) Batch processing — when processing multiple items through the same tool pipeline. (5) Token-cost-sensitive workloads — when token cost is a primary KPI. When NOT to use code execution orchestration: (1) Simple workflows — 1-3 tool calls. The overhead of writing a script is not worth it. (2) Interactive workflows — when the user needs to see intermediate results and provide feedback between steps. (3) HITL workflows — when human approval is needed between tool calls. (4) Dynamic tool selection — when the LLM needs to dynamically decide which tool to call next based on intermediate results. Security considerations: (1) Sandboxed execution — the Code Execution tool must be sandboxed. No network access (or restricted access), no production credentials, minimal filesystem permissions. (2) Strict allowlist — validate the script against a strict allowlist of permitted operations before execution. (3) Output scanning — scan the script's output for known malicious patterns (shell injection, SQL injection, SSRF). (4) Resource limits — set CPU, memory, and time limits on the sandboxed execution. (5) No production credentials — the script runs with minimal privileges. Tools are called via the sandbox's restricted API, not with production credentials. Implementation: (1) Claude Agent SDK — implements this pattern natively. Claude writes Python scripts that orchestrate workflows in a sandboxed Code Execution tool. (2) Custom implementation — provide a Code Execution tool that accepts Python scripts, runs them in a Docker container or VM with restricted permissions, and returns the output. (3) LangGraph — implement as a custom node that executes code. Use LangGraph's checkpointing to persist state between code execution steps. The key: 'Anthropic's code execution pattern demonstrates a 98.7% token reduction by having agents write orchestration scripts instead of making individual tool calls.' For complex multi-tool workflows, code execution orchestration is the most token-efficient approach. The agent writes a script, the script runs in a sandbox, and only the final result enters the LLM's context. This eliminates the token overhead of intermediate tool calls and results (chatforest 2026, Anthropic 2026).

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →