Multi-Agent Orchestration That Actually Scales
Config-driven multi-agent systems beat hard-coded chains. Lessons from routing requests across dozens of specialist agents — when to add an agent, and when not to.
Most "multi-agent" systems are a single prompt wearing a costume. Real orchestration earns its complexity only when work genuinely fans out to specialists and the routing between them is dynamic. Here's what held up building one that routes across dozens of agents.
Routing is the product
The agents are the easy part — each is a focused prompt plus tools. The hard, valuable part is deciding which agent handles a request, in what order, and when to stop. Get routing wrong and 42 great agents produce confidently wrong answers fast.
Config over code
Hard-coding the graph means every new capability is a code change, a review, a deploy. Declaring agents and routes in YAML means adding a specialist is a config edit. The orchestrator reads the config and builds the graph at runtime.
- Each agent: a name, a role prompt, its tools, and the conditions under which it's eligible.
- Routing rules live beside the agents, not buried in control flow.
- New department? New YAML block. No orchestrator rewrite.
Conditional routing, not fixed chains
A fixed A→B→C chain wastes calls when B wasn't needed and breaks when the request doesn't fit the chain. Conditional routing — the kind LangGraph models cleanly — sends each request only through the nodes it needs, with explicit edges for the branches.
Add an agent when a request type is consistently handled badly by every existing one. Not before.
The failure modes nobody warns you about
- Loops. Agents handing work back and forth. Cap hops and make the stop condition explicit.
- Lost context. Each hop is a chance to drop state. Pass a structured scratchpad, not free text.
- Cost blow-ups. Every hop is a model call. Route to fewer, better-targeted agents.
Takeaways
- Spend your design budget on routing, not on more agents.
- Declare the system in config so capabilities are edits, not deploys.
- Prefer conditional graphs over fixed chains; cap hops and define the stop.
- Only add an agent that closes a real, repeated gap.