跳转至

Routing: Choose The Path Before Running It

A travel assistant eventually receives very different inputs:

  • "Calculate whether 600 RMB is enough for two people."
  • "Make this itinerary sound more natural."
  • "Check tomorrow's weather in Hangzhou."

If every request goes through one giant prompt, the assistant becomes broad and sloppy. Routing adds a decision point first: should this input go to math, writing, retrieval, or general answer?

One Sentence

Routing turns one universal flow into route selection plus specialized flows, so tools, budgets, and safety policies can be narrowed by route.

What Breaks Without It

Problem What it looks like Risk
One assistant handles everything Simple entrypoint Too many tools, wrong choices
Writing tasks get retrieval tools Flexible More cost and irrelevant evidence
Math goes through long-form generation It can answer Slow, expensive, harder to test

What This Pattern Changes

Who Owns
Router Chooses a route id like math, writing, research
Python Runs the workflow / agent / toolset bound to that route
Route Uses narrower prompts, tools, and budgets

Route ids are contracts. Keep the set small and stable.

Walk Through One Trace

Input Router output Flow Why
Compute 2+2. math Math flow Contains calculation intent
Write a short poem about the ocean. writing Writing flow Text generation task
Will it rain in Hangzhou tomorrow? research/weather Retrieval or weather tool Needs external facts

Flow

flowchart TD
  U["User input"] --> R["Router"]
  R -->|math| A["Math workflow"]
  R -->|writing| B["Writing workflow"]
  R -->|research| C["Retrieval / tool route"]
  A --> O["Output"]
  B --> O
  C --> O

Code Walk

Rule routing is cheap:

picked1 = rule_route(
    text1,
    rules=[
        Rule(route="math", predicate=lambda t: any(ch.isdigit() for ch in t)),
        Rule(route="writing", predicate=lambda t: "poem" in t.lower()),
    ],
    default="general",
)

LLM routing is more flexible, but should return structured output:

picked2 = llm_route(
    model,
    text=text2,
    routes=[
        Route(name="math", description="Solve math problems"),
        Route(name="writing", description="Write or edit text"),
    ],
    tracer=tracer,
)

Full example:

from __future__ import annotations

from pathlib import Path

from agent_patterns_lab.patterns.routing import Route, Rule, llm_route, rule_route
from agent_patterns_lab.runtime import MockLLM, Tracer


def main() -> None:
    tracer = Tracer()
    model = MockLLM(['{"route":"writing"}'])

    text1 = "Compute 2+2."
    picked1 = rule_route(
        text1,
        rules=[
            Rule(route="math", predicate=lambda t: any(ch.isdigit() for ch in t)),
            Rule(route="writing", predicate=lambda t: "poem" in t.lower()),
        ],
        default="general",
    )

    text2 = "Write a short poem about the ocean."
    picked2 = llm_route(
        model,
        text=text2,
        routes=[
            Route(name="math", description="Solve math problems"),
            Route(name="writing", description="Write or edit text"),
        ],
        tracer=tracer,
    )

    print({"rule_route": picked1, "llm_route": picked2})

    trace_path = tracer.export_jsonl(Path(".traces") / "12_routing.jsonl")
    print(f"[trace] {trace_path}")


if __name__ == "__main__":
    main()

Run:

UV_CACHE_DIR=.uv_cache PYTHONPATH=src uv run --no-sync python examples/12_routing.py

Nearby Patterns

Pattern Who decides next Use when
Prompt Chaining Python fixes order One task type, fixed steps
Routing Router picks a path Inputs are different types
Handoff Current agent transfers ownership You discover the wrong owner mid-run
Agents-as-Tools Controller calls specialist agents Controller keeps final-answer ownership

When To Use It

  • Inputs fall into a few stable categories.
  • Categories need different tools, models, budgets, or guardrails.
  • You want route choice in the trace.
  • Misroutes can be measured with a small eval set.

When Not To Use It

  • There is only one path.
  • Categories are fuzzy even for humans.
  • The right owner is only discovered mid-run; that is handoff.
  • You have no default route or clarification path.

Costs And Common Failures

Failure Symptom Fix
Misroute Writing task goes to math Add confidence, fallback, routing evals
Too many routes Router becomes confused Merge low-value routes
No audit Only final answer is logged Trace route id and reason
Tool leakage Every route sees every tool Bind tool allowlists per route

Routing is entrypoint dispatch, not multi-agent collaboration by itself.

If ownership transfers mid-run, read Handoff. If a controller calls experts without transferring ownership, read Agents-as-Tools.

References