Multi-Server MCP Architecture: Routing, Isolation, and Control

Core idea: connecting more servers should increase capability without collapsing identity, permissions, failures, and tool names into one undifferentiated pool.

A serious agent host may connect to separate MCP servers for repositories, tickets, documents, analytics, and deployments. The protocol gives each client-server relationship a consistent shape. The host still needs architecture for discovery, routing, policy, lifecycle, and observability across the fleet.

TL;DR

• Maintain one isolated client context per server, credential set, and trust boundary.
• Build a canonical registry that preserves server provenance and resolves name collisions.
• Filter capabilities by user, tenant, environment, and task before model exposure.
• Route through deterministic policy; do not let the model choose credentials or endpoints.
• Degrade per server so one unhealthy integration does not disable the whole agent.

Reference architecture

The host owns a server manager, client connections, capability registry, policy engine, model adapter, result normalizer, and telemetry pipeline. Each configured server has endpoint or launch details, credentials, protocol compatibility, health state, limits, and an owner.

On request, the host determines user, tenant, environment, and task. It selects eligible servers, refreshes discovery when required, filters capabilities, maps them into the model interface, validates the chosen call, routes it through the correct client, and normalizes the result.

The model sees a deliberately constructed capability surface—not every tool every server happens to advertise.

Names and provenance

Two servers may expose search, getstatus, or createticket. Internally use a stable identity such as (serverid, capabilitytype, capabilityname) and map it to a model-safe name like githubissue_search. Preserve the original name for the protocol call.

Descriptions should state domain, boundary, and important exclusions. Avoid silently renaming tools in a way that breaks audit trails. Registry records should include server version, discovery timestamp, schema hash, risk tier, required scopes, and status. Unexpected schema or metadata changes deserve review.

Policy-aware discovery

Discovery is not authorization, but it shapes model behavior. A production deployment tool should be absent for a user who cannot deploy. A customer-support task should not include repository administration merely because the same host can connect to that server.

Filter first by organizational configuration, then identity and tenant, environment, task relevance, and risk. Recheck immediately before execution because permissions or state may change after discovery. Keep approval rules deterministic and specific to the final arguments.

Routing and isolation

The router resolves the model-facing name to exactly one server and capability. It attaches the correct authenticated context, applies deadlines and concurrency limits, and never accepts a model-provided endpoint or credential.

Isolation limits blast radius. Use separate processes or containers for local servers with different trust levels. Use separate tokens and connection pools for remote domains. A compromised document server should not inherit deployment credentials. Prevent tool results from one tenant entering another tenant's model context or cache.

A concrete workflow

A user asks: “Check the failed checkout deployment, find the related incident, and draft an update.” The host exposes read-only deployment status, incident lookup, and draft-message tools.

The model calls the deployment server through its scoped client. The host normalizes the result and passes only necessary fields back into context. It then routes the incident identifier to the ticket server after tenant validation. Finally, it creates a draft through the communications server. Sending the message is a separate, approval-gated capability.

The workflow crosses three servers, but identity, correlation, and policy remain continuous. No server receives credentials belonging to another.

Failure management

Track health independently. If the ticket server is unavailable, return a partial result and a clear next action rather than claiming the whole task failed or retrying every server. Use per-server circuit breakers, backoff, quotas, and deadlines.

Discovery may be cached, but define freshness. Refresh after server upgrades, permission changes, list-changed notifications where supported, or repeated unknown-capability errors. Do not silently fall back to another server with broader permissions merely because the preferred one is unhealthy.

Observability

Create one logical operation ID spanning model turns and server calls. Record user and tenant pseudonyms, exposed capability set, selected server and tool, policy decision, approval, attempt, latency, result class, and downstream references. Metrics should show per-server health, discovery failures, collisions, denials, tool-selection frequency, timeouts, and cost.

Keep full sensitive payloads out of general traces. Preserve provenance so a surprising result can be tied back to its server, version, and schema.

Common mistakes

• Flattening tools without server provenance.
• Sharing one administrator credential across servers and users.
• Exposing the complete registry to every task.
• Letting the model select endpoints or authentication.
• Treating cached discovery as permanent authorization.
• Making one server failure fatal to the whole host.
• Falling back to a more powerful server without consent.

My Take

The host is the control plane of multi-server MCP. Its job is not merely to aggregate tools; it must preserve boundaries while composing useful work. A smaller task-specific registry with explicit provenance usually produces better model decisions and safer operations than an impressive wall of available tools.

Sources

• Model Context Protocol specification: Architecture, capability discovery, and per-request metadata
• Model Context Protocol specification: Tools, resources, lifecycle, errors, and authorization