Routing requests to specialists and providers

routing only: dispatch passes an answer through without synthesis

An agent team is declared in one team definition. The definition names an orchestrator and the sub-agents it can call, and gives each agent its instruction, its tools and the model it runs on, along with the memory tier the team keeps. By default the team runs by delegation: the runtime wraps each sub-agent as a tool on the orchestrator, and the orchestrator’s model decides which ones to call.

Let us review the standard delegation cycle, focusing on its final turn. The sub-agent’s response arrives as a tool observation, prompting the orchestrator to generate a concluding response. When the orchestrator genuinely synthesizes information (querying multiple specialists, reconciling conflicting data, and assembling a unified answer), this final turn represents critical reasoning. Delegation was built for that case.

However, many production systems follow a simpler pattern. Every incoming user query maps to exactly one specialist, each specialist generates an autonomous, comprehensive response, and the orchestrator’s system instruction is a pure router’s: pass the specialist’s answer directly to the user without summarizing, altering, or editing it.

Consider what the orchestrator’s final turn actually does in that architecture. It spends an entire model inference call merely relaying text it has been explicitly forbidden to modify. Because this relay is itself a generative model turn, it introduces unnecessary failure risks. In real production deployments, this relay pattern exhibited two severe failures over two days: in one instance, the orchestrator consumed zero generation tokens and returned an entirely blank string to the user; on the following day, it emitted the literal string FinancialAnalyst instead of forwarding the specialist’s 2,599-character analysis. Although the prompt strictly forbade editing, system instructions cannot guarantee faithful verbatim reproduction, because token generation is inherently probabilistic.

look first: count the model calls in each lane, then find the one that changes nothing.

A team definition can declare an alternative coordination strategy for this case. When the definition includes a dispatch: configuration block, the team switches from delegation to plan-dispatch:

dispatch:
  default_route: "Generalist"   # the fail-static target: a declared
                                # specialist that can handle ANY message

orchestrator:
  name: "Triage"
  instruction: |
    Name exactly ONE specialist for this message. ...
  output:                       # the classifier's whole answer:
    schema:                     # one JSON object, read by code
      type: "OBJECT"
      properties:
        route:  { type: "STRING", description: "Exact specialist name" }
        reason: { type: "STRING", description: "≤8 plain words for the waiting user" }
      required: ["route"]

Read that as: the orchestrator’s whole output is now one JSON object with a required route naming a specialist and an optional reason for the user to read while they wait; and if that object is unusable for any reason, the request goes to Generalist.

Under plan-dispatch, the orchestrator has no attached sub-agent tools. It acts as a pure classification model: it evaluates the incoming message and emits approximately 15 tokens of JSON specifying the target route. Host application code parses that specialist name, invokes the selected specialist directly, and returns that specialist’s raw output straight to the user. The fragile relay turn is completely eliminated because there is no intermediary model step between the specialist and the end user. This represents a structural fix rather than an iterative patch: instead of retrying or gating a fragile relay step, we eliminate the step entirely.

	delegation (the default)	plan-dispatch (dispatch: declared)
sub-agents are	tools on the orchestrator	plain definitions code selects from
the routing decision is	implicit in which tool the model calls	an explicit value in code
the final answer comes from	the orchestrator, re-emitting it	the specialist itself
model calls per request	classify + specialist + relay	classify + specialist

Here the schema on the classifier does the structural work. The classifier holds no sub-agent tools, so its schema-bound JSON is the whole of its turn, and the hand-off moves out of the model and into deterministic code. That is precisely where routing logic belongs. When a routing decision exists as an explicit program variable, your application code can log it, trace it in telemetry, test it in unit suites, and stream it to the user as real-time feedback (such as “Routed to Analyst — chart question”). Under delegation, that same decision remains an opaque tool call buried deep within internal execution traces.

Handling routing in code also makes error recovery deterministic. Consider the possible failure modes: if the orchestrator produces malformed JSON, code catches the parse error and routes to default_route; if the classifier outputs a non-existent agent name, the lookup fails and falls back to default_route; if the model errors out or returns an empty payload, the handler catches the exception and invokes default_route. Every failure branch resolves safely to a verified specialist:

Every routing failure resolves to the default route: whatever goes wrong while the route is being chosen, the resolver selects default_route, the one specialist declared able to handle anything. Malformed JSON, an unknown specialist name, an empty reply, a classifier that errors outright: all of them dispatch to that specialist. The guarantee covers the choice of route. If the chosen specialist then fails, because its provider errors or its run is stopped, the turn ends failed and the user receives an error message naming that route; a route that returns empty text produces a notice naming it.

You cannot achieve this level of reliability through prompt engineering alone, because prompts only influence model behavior when the model generates valid tokens. A deterministic fallback function, tested against arbitrary malformed inputs, guarantees that the system always resolves to a valid, declared specialist.

What failure modes remain? The first is a route that fails after it is chosen. The deterministic fallback guarantees that routing resolves to a declared specialist; it does not run a second specialist when that one fails. Before the failure reaches the user, the engine retries transient HTTP failures, three attempts in total, and an agent that declares fallback_model retries a provider-side failure on a second model; a failure that survives both ends the turn with an error message naming the route. The second is route quality. A deterministic fallback ensures a declared specialist is invoked, but that specialist may be the fallback Generalist when a dedicated specialist would have been preferable. Similarly, if the classifier confidently chooses the wrong specialist, the fallback resolver cannot detect the mistake. This failure mode mirrors misread tool descriptions in delegation mode and must be addressed by refining the classifier’s prompt and reviewing production routing logs.

While plan-dispatch improves reliability, it introduces session management challenges that delegation handles automatically. Under delegation, the entire interaction takes place within a single unified session graph, allowing the conversation transcript to maintain its own continuity. Under plan-dispatch, no automatic threading exists, requiring careful manual session orchestration.

The first issue involves message isolation. The orchestrator outputs JSON routing decisions; if those raw verdicts entered the shared conversation session, subsequent specialists would interpret internal routing metadata as user dialogue. To prevent this, the classifier is isolated from writing to the conversation history, while all downstream specialists write to a single shared session.

However, simply sharing a storage transcript does not ensure conversational coherence across different specialists. When the runner formats stored conversation turns into a prompt, it inspects message authorship relative to the agent currently executing. In delegation mode, the orchestrator authors every top-level assistant turn throughout the entire session. In dispatch mode, the author changes with every routing transition. When a second specialist takes over, the previous specialist’s response is no longer recognized as the current agent’s output; the engine treats it as foreign speech and quotes it into the prompt as user dialogue. Across an extended interaction, a specialist can receive the entire conversation history as an undifferentiated block of user messages, with no turns identified as assistant speech.

In a live 4-turn interaction routed across 3 distinct specialists, replaying the raw session through the standard prompt builder produced 15 messages totaling 118,000 bytes for the final specialist, and every single message was labeled as user dialogue. The recorded history was technically complete, but its conversational structure was entirely lost. This raw transcript swept in internal reasoning tokens from prior specialists alongside raw tool outputs, including a 23,000-character JSON payload forwarded to a lightweight conversational agent with no tools and a 400-character response budget.

Sharing a session is not sharing a conversation. Continuity across agents needs a stored turn to arrive as a turn: attributed, in the assistant’s voice, without the reasoning and tool traffic that produced it. Storage alone gives you none of that.

The solution is a read-time session projection. When reading history for an upcoming specialist, previous assistant turns are re-attributed to the active agent so the engine renders them correctly as assistant dialogue. The name of the specialist who originally authored each response is preserved as a concise metadata label, and all internal thinking tokens, intermediate tool invocations, and raw tool payloads are filtered out. Downstream specialists only need the final synthesized answers. The underlying database still stores the complete, unaltered interaction for long-term vector ingestion, but the projected context presented to the active model reduces that same thread to 8 clean, alternating user-assistant messages totaling under 8,000 bytes.

The orchestrator requires the inverse configuration. While it must never write its internal JSON verdicts into the conversation transcript, it must be able to read recent history to distinguish contextual follow-ups from standalone queries. Without conversational context, a user message like “you’re neglecting where the world is going” gets misclassified as casual small talk because the classifier cannot see the preceding analysis. To solve this, the classifier receives a condensed digest of recent alternating turns positioned immediately above the target user message.

In summary: plan-dispatch delivers reliable, deterministic routing hand-offs, but it requires explicit session projection and context management that delegation provides natively. Proper session engineering goes well beyond merely keeping routing verdicts out of the primary transcript.

Choosing between these two patterns comes down to evaluating the orchestrator’s role:

Dispatch routes; delegation composes: dispatch is for choosing a specialist, delegation for combining several. An orchestrator that adds nothing to a specialist’s answer should not be in the reply path at all, and an orchestrator that genuinely composes is not a relay, so delegation remains exactly right for it.

when the routing rule is code: a script holds the logic between agents

Delegation and plan-dispatch both leave the logic between agents to a model. Under delegation the orchestrator’s model decides each hand-off; under plan-dispatch a classifier names one route, and code the team cannot change runs it. A third method, the workflow graph, fixes the order of the steps in the definition and lets no model choose it. Some routing mixes fixed rules with a model’s judgment: a pattern in the message that should always win, a classifier consulted only when no pattern matches, a cheap model for one call and a stronger one for the next, a question to the person halfway through. Written as a prompt, a rule like that is a request the model may not follow. Written as engine code, it changes only with an engine release.

The fourth method puts that logic in the team definition as a short program. The script is one JavaScript module. It receives the message, a digest of the conversation before it, and a set of functions: agent runs one of the team’s declared agents on an input and returns its answer, tool runs a tool the definition allows, askUser puts a question to the person, and parallel runs several calls at once. Each agent call can choose its own model, reasoning effort and temperature from a list the definition declares. The agents keep their instructions, tools and schemas; the orchestrator becomes one more agent the script may call.

script:
  models: ["gemini-3.8-flash", "claude-opus-5-5"]
  source: |
    export default async ({ message, agent, log }) => {
      if (/\b(chart|candles?)\b/i.test(message)) return agent("Analyst", null, { conversation: true });
      const triage = await agent("Triage", message);
      log(`Routed to ${triage.route}`);
      return agent(triage.route, null, { conversation: true });
    };

Read that as: a message that mentions a chart goes to the analyst without a model call; any other message goes to the classifier, the progress line names the route it chose, and the route answers in the conversation. Here the routing rule is a line of code, and a change to it deploys with the file.

	the orchestration lives in
delegation (the default)	the orchestrator’s model, choosing tools
plan-dispatch	one classifier call; code runs its route
workflow graph	the order of the declared steps
script	a program in the definition, calling agents as functions

A script reaches the world only through the functions it is given. It runs in a sandbox, an interpreter with no network, no file system, no clock and no randomness of its own. Its clock and its random numbers come from the engine. That restriction is what makes a paused script resumable. Every call the script makes, and the result each call returned, is written to a journal in the conversation. When the script asks the person a question, the turn ends and waits. The person’s reply runs the same script again from the top: each call already in the journal is answered from it, the open question takes the reply as its answer, and the calls after it run live. The paused run keeps the source it started with, so a redeploy never breaks a conversation that is waiting.

The same pause also gates a tool. A tool call the definition marks for approval waits for a person to approve that exact call; approved, it runs with the arguments the person saw, and refused, the script receives an error it can handle. A script team can also sit inside another team, as a specialist, a route or a step in a workflow graph, and a question it asks pauses the outer turn until the answer comes. The trace records each run and each call by kind and name, never by its arguments or its result.

A script makes the same calls on every run: replay matches each call against the journal by its kind, its name and its arguments. A script that branches on something outside its input, or calls in a different order the second time, no longer matches, and the resumed turn fails with an error instead of guessing.

The engine also gives the sandbox to agents as a tool. An agent given the code tool writes a JavaScript module, the engine runs it, and the agent reads back the returned value, the logged lines and any error before it answers. Because the model writes only text, the tool works on every provider. A provider’s own code execution, such as Gemini’s Python sandbox, works only on that provider’s models. A run that stops on an error returns the error, and the model may correct the code and run it again; a run cannot reach the network or the host, and it stops at fixed memory and compute limits. The code and its result can also travel with the reply, so a reader can check the computation the answer rests on.

a model-string prefix routes each agent to its provider

The model registry, which routes each request by inspecting the prefix of the model string, is what makes multi-provider agent graphs practical. Because each agent declares its own model: string, provider selection is defined per agent:

• gemini-* → Google’s native API, which also receives any model string the registry does not recognize
• claude-* → the Anthropic adapter
• gpt-* and the o-series (o followed by a digit) → the OpenAI adapter
• grok-* → the xAI adapter
• ollama/* → your local runtime, executed without external network requests

Read that as: the prefix of one string, on one agent, decides where that agent’s arithmetic runs, and no other agent in the file has to agree with it.

Each cloud provider expects its own key, and a team that mixes three providers would ordinarily need three. A model gateway, a service such as Vercel’s AI Gateway or OpenRouter that fronts many providers behind one key, removes that need: once a gateway is configured, the registry serves every model whose direct key is absent through that gateway. A direct key, whenever it is present, takes precedence, and its agents return to the provider’s own adapter. The gateway path drops provider-side tools such as native web search, so an agent that relied on one loses it when its direct key is missing.

This enables practical hybrid architectures: a team can run a local open-weight model over sensitive on-premise documents while utilizing a frontier cloud model for high-level synthesis on non-sensitive summaries. The privacy boundary is configured per agent, on the same line that names the model.

A team does not have to run every agent on the same model or provider.

serving the agent graph

Any team definition can be hosted directly as a networked service, an Agent-to-Agent (A2A) endpoint. Remote applications and external agent networks can interact with the graph via standard HTTP requests. The service automatically publishes an Agent Card (a structured, machine-readable manifest describing the team’s capabilities) allowing external callers to inspect available functions programmatically. The server uses the caller’s contextId as the session id, so a client continues a conversation by sending the same one, and a task that waits for a human approval ends in the input-required state.

Here is the complete lifecycle of a team’s turn, summarized in the core concepts we have established:

1. LOAD
   The framework reads the team definition: orchestrator,
   sub-agents, each one's instruction, tools, model, config.

2. WRAP
   Each sub-agent becomes a tool on the orchestrator. Its
   description is the tool's description.
   (dispatch: declared → skip this; sub-agents stay plain
   definitions that code will select from)

3. TURN
   The orchestrator's model reads its instruction and every
   description, and emits tool calls, or an answer.

4. RUN
   The runner executes each call. A sub-agent call runs that
   agent's whole loop (its own turns, its own tools) and returns
   its text as the observation. Every model call goes through the
   registry, by prefix.

5. LOOP
   Back to 3 while calls continue. When the orchestrator writes
   an answer instead, the turn ends.
   (dispatch: the classifier's JSON names one route; code runs
   that specialist; its output is the answer; any routing
   failure → default_route; a route that fails → an error)
   (script: the definition's program calls agents and tools
   as functions; its return value is the answer; a question
   to the person pauses it, and the reply replays the journal)

A schema sits on the agent whose judgment is the answer:
a specialist leaf, an orchestrator that delegates,
or the classifier under dispatch:.

what you can do with this today

Read a system before you run it. Because the architecture is fully declarative, the entire topology is captured in one YAML file. Practice inspecting team definitions before running them (including your own past projects) and identify the structural boundaries governed by each configuration key.

Put the schema on the agent whose answer it shapes. A specialist whose answer the user should receive holds its schema as a leaf. An orchestrator whose own verdict is the answer may hold the schema beside its sub-agents; check which provider path that orchestrator runs on first, because on a degraded path the schema reaches the model as a tool it can skip, and validate the JSON where your code consumes it, on either path. An orchestrator that only routes becomes a plan-dispatch classifier.

Tune hyperparameters per role, not per system. Different roles require different operational parameters. Use sampling: and reasoning: to tailor thinking budgets, output token limits, and sampling parameters to each specialist’s specific workload rather than imposing global system defaults.

Ask what the orchestrator adds. If an orchestrator simply echoes a single specialist’s findings, it acts as an unnecessary relay. Configure dispatch: to eliminate the relay step from the execution path. If the orchestrator actively reconciles and synthesizes multiple inputs, retain standard delegation. If the routing rule is something you would write as code, a pattern that always wins or a model chosen per call, write it as a script.

Ask where each agent’s arithmetic runs. Review the model: prefixes throughout your team definition. Any specialist handling sensitive data can be pointed to a local ollama/ instance without altering the rest of your agent graph.

In our next lesson, we explore memory architectures: managing session state that persists across restarts, and building long-term memory systems that distill conversations into durable, retrievable facts. We will build all three persistence tiers governed by memory_system. Proceed to building memory systems for agents to continue.