Agent Routing Patterns: How Agents Choose the Next Worker

An agent router answers a bounded question: given this request and the relevant state, which registered destination should receive the work?

That destination might be a specialist agent, workflow, model, tool family, human queue, or fallback. The routing mechanism can be as simple as an if statement or as flexible as an LLM. The correct choice depends on ambiguity, capability overlap, cost, and the consequences of a mistake.

This article explains seven routing patterns and the controls that keep routing from becoming an expensive, opaque agent of its own.

TL;DR

• Routing selects a destination; it does not automatically coordinate the downstream workflow.
• Prefer deterministic rules when signals and policies are explicit.
• Classifiers and semantic routers work well for stable categories expressed in varied language.
• LLM routers are useful for nuanced intent, but need bounded outputs, confidence handling, evaluation, and fallback.
• Overlapping capabilities, routing loops, and missing defaults cause more failures than the route-selection algorithm itself.

The core router architecture

A router should emit a bounded destination and confidence signal; it does not automatically own the downstream workflow.

An [agent router](/glossary/agent-router/) has five core elements:

1. Input: request plus selected state.
2. Signals: intent, language, risk, data type, user tier, or current workflow position.
3. Destinations: a registered set of agents or paths with clear capabilities.
4. Decision: route identifier, confidence, and reason codes.
5. Fallback: behavior for ambiguity, low confidence, unavailable destinations, or invalid output.

A router may normalize the request or redact sensitive data, but its primary responsibility is selection. Once transfer succeeds, it may exit.

Seven routing patterns

1. Rule-based router

Rules select a route from explicit fields:

• file_type == "csv" → data workflow;
• risk_level == "high" → human review;
• command begins with /code → coding agent;
• authenticated region determines compliance path.

Use rules when policy is stable and the signals are reliable. Rules are fast, inexpensive, and easy to audit.

They become difficult when natural-language categories overlap or when hundreds of special cases accumulate. A rule engine should still have a default route and conflict precedence.

2. Classifier router

A classifier maps an input to one of a known set of labels. It may be a traditional machine-learning model, a small language model, or a larger model constrained to structured output.

Use it when categories are stable, examples are available, and language varies. Support routing is a common case: billing, technical issue, account security, or general question.

Measure per-route precision and recall rather than only overall accuracy. A classifier that is excellent on the common “general” class may still misroute rare security requests.

3. LLM router

An LLM router reads natural-language capability descriptions and chooses a destination. It handles nuanced requests and can consider several signals without a separately trained classifier.

Use it when intent is ambiguous, route descriptions change, or the decision needs language understanding that rules cannot express economically.

Constrain the output to registered route IDs. Validate the schema. Use a small, fast model when it meets the routing quality target. Do not allow the model to invent agent names, tools, or URLs.

4. Semantic router

A semantic router represents the request and route examples or descriptions as embeddings, then selects the closest capability by similarity.

Use it when routes correspond to recognizable meanings expressed in many phrasings. It can be fast and cheaper than a generative router.

Similarity is not certainty. Two routes may both be close, and a request may be unlike every example. Calibrate thresholds, inspect nearest alternatives, and send low-confidence cases to a fallback.

5. Capability router

A capability router matches task requirements against a registry of what each destination can do. The registry can include:

• supported task types;
• tools and data access;
• input and output schemas;
• languages or domains;
• cost and latency class;
• security clearance;
• current availability.

Use it when destinations change dynamically or when route eligibility depends on permissions and operational status, not only intent.

Capability routing should separate can handle from is best. First filter ineligible destinations deterministically; then rank the eligible set.

6. Hierarchical router

A hierarchical router makes decisions in stages. The first stage might choose a domain; a second router chooses a specialist inside that domain.

For example:

Request → Operations → Finance Operations → Invoice Reconciliation Agent

Use it when one flat label set is too large or when teams own their own route policies. Hierarchy reduces each decision’s choice set, but early mistakes propagate. The parent route should support escalation when no child fits.

7. Router plus fallback

Fallback is not a separate algorithm; it is an architectural pattern that makes uncertainty explicit.

A router may:

• send low-confidence requests to a general agent;
• ask the user one clarifying question;
• route high-risk ambiguity to a human;
• try a deterministic backup;
• decline unsupported requests;
• retry after a capability registry refresh.

Every production router needs a defined fallback. “Pick the closest route anyway” converts uncertainty into silent misrouting.

A testable routing pipeline

Separating signal extraction, capability matching, confidence, and fallback makes routing decisions testable.

Treat routing as a pipeline:

1. Extract signal: normalize the request and identify task, domain, risk, language, and state.
2. Match capability: filter and rank registered destinations.
3. Check confidence: compare the best route to thresholds and alternatives.
4. Route or fall back: produce a validated transfer decision.

This separation helps diagnose errors. If a refund request goes to technical support, did signal extraction miss “charged twice,” did route descriptions overlap, or was the confidence threshold too low?

Record the input version, candidate routes, selected route, confidence, reason code, router version, and fallback outcome. Do not store hidden model reasoning as the decision contract.

Router vs orchestrator

A router selects a destination. An [orchestrator](/glossary/orchestrator/) coordinates execution over time: dependencies, state, retries, waits, aggregation, and completion.

A router can start an orchestrated workflow. An orchestrator can contain routing decisions at several graph nodes. The responsibilities can live in the same component, but they remain different.

[Orchestrator vs Supervisor vs Router](/orchestrator-vs-supervisor-vs-router/) provides the full role comparison.

Router vs supervisor

A [supervisor agent](/glossary/supervisor-agent/) manages workers and remains responsible for progress. It may assign, review, request revision, and aggregate.

A router normally makes one selection. If a component repeatedly observes worker results and chooses the next assignment, it is performing supervision—not just routing.

Routing vs delegation

[Delegation](/glossary/delegation/) assigns a bounded task while the delegator retains overall ownership. Routing identifies where work should go. A supervisor may route a subtask to the best worker as part of delegation.

The route decision does not define the ownership contract. [Agent Handoffs, Delegation, and Sub-Agents](/agent-handoffs-delegation-sub-agents/) explains how ownership, context, permissions, and expected results should transfer.

Routing vs tool selection

[Tool calling](/glossary/tool-calling/) chooses a capability to invoke, usually for a bounded operation. Routing chooses a component or workflow path.

The distinction can blur when specialist agents are exposed as tools. Architecturally, ask:

• Does the caller wait for a bounded result and keep ownership? That resembles tool use or delegation.
• Does the destination become the active owner? That resembles routing plus handoff.

Implementation syntax should not decide the product-level ownership model.

Routing vs planning

[Planning](/glossary/planning/) determines how to achieve a goal, often by creating or revising multiple steps. Routing selects a destination for the current request or state.

A planner may produce tasks that are later routed. A router should not invent a multi-step plan unless that responsibility is explicitly part of its contract.

When an LLM router is unnecessary

Do not use an LLM when:

• an explicit field determines the route;
• policy mandates a destination;
• there are only two clean cases;
• keywords or schemas separate inputs reliably;
• latency or cost is critical;
• the decision must be exactly reproducible;
• a security boundary requires deterministic enforcement.

Start with rules. Add a classifier or semantic model when language variation defeats them. Add an LLM only when measured errors justify the additional cost and variability.

A useful hybrid applies deterministic eligibility filters first, then uses a model to rank only allowed destinations.

Routing failure modes

Wrong classification

The selected route cannot handle the task. Validate with labeled examples and downstream outcome metrics, not only route accuracy.

Ambiguous intent

The request reasonably fits several destinations. Ask for clarification or choose a general path instead of forcing confidence.

Overlapping capabilities

Agent descriptions use vague phrases such as “handles analysis.” Define boundaries, exclusions, examples, and required inputs.

Routing loops

Agent A routes to Agent B, which routes back to A. Track visited routes, limit transfers, and define an escalation destination. An explicit [agent graph](/agent-graphs-and-state-machines/) can make allowed transitions visible.

Excessive router complexity

The router evolves into a planner, policy engine, registry, supervisor, and recovery runtime. Split responsibilities and keep route output small.

No fallback

Invalid or low-confidence decisions silently go to the closest specialist. Make uncertainty a first-class outcome.

Expensive routing model

The router costs nearly as much and takes nearly as long as the downstream task. Test smaller models, classifiers, embeddings, or rules.

Practical example: analytics assistant

Suppose a user asks, “Compare churn for enterprise customers this quarter and explain the likely drivers.”

A robust router might:

1. extract analytics intent and enterprise/customer segments;
2. confirm the user is permitted to access the dataset;
3. identify the data-analysis workflow as eligible;
4. detect that explanation requires both querying and interpretation;
5. route to the analytics workflow with a confidence score;
6. use the general agent only if the data workflow is unavailable.

The router does not design the SQL, execute tools, interpret results, or write the answer. Those are downstream responsibilities.

When not to route to another agent

If one agent already has the necessary tools and instructions, routing may add no value. A new agent is justified when it creates a meaningful boundary: different permissions, specialized context, distinct evaluation, separate ownership, or a different workflow.

Do not create a route for every tiny capability. That turns the router into a fragile menu and fragments context across artificial roles.

My take

The best router is often less intelligent than the agents it routes to.

Its job is to make a small, observable, reversible decision. Deterministic eligibility, bounded outputs, calibrated confidence, and an honest fallback matter more than making the router sound agentic.

Sources

• Building Effective AI Agents defines routing as classification followed by a specialized task and recommends it when distinct categories can be classified accurately.
• LangGraph workflows and agents demonstrates structured-output routing and conditional graph edges.
• Agent orchestration in the OpenAI Agents SDK describes both model-led selection and code-led classification into bounded next steps.

Continue learning

Place routing inside the wider [Multi-Agent Coordination Patterns](/multi-agent-coordination-patterns/) map, then compare how routing authority changes in [Centralized vs Decentralized Multi-Agent Architectures](/centralized-vs-decentralized-multi-agent-architectures/).