Pattern 02 · AI Architecture
Multi-Agent Systems that stay controllable
A production pattern for decomposing complex work across specialist agents — with planning, shared state, policy gates, and human approval where the stakes demand it.
01 · PROBLEM
Single agents collapse on multi-step work
A single agent with a long prompt works until the task needs real decomposition: research a topic, analyze data, write code, then verify the result. Context windows fill with intermediate state, tool-call sequences get confused, and there's no separation between "doing the work" and "checking the work." The failure mode is confident, expensive wandering.
This pattern covers orchestrated specialist agents for workflows like incident triage, research synthesis, data-analysis pipelines, and software tasks — where planning, execution, and verification are distinct jobs with distinct tools and distinct risk profiles.
02 · WHEN TO USE
Fit and anti-fit
- Tasks decompose into 3+ distinct phases with different tools (search → analyze → act → verify).
- Verification must be independent of execution (the builder shouldn't grade its own work).
- Some steps are high-stakes (write to prod, send to customer) and need approval gates.
- You can define task-level success criteria for evaluation.
Don't use it when a single agent with tools passes your evals — every additional agent adds latency, cost, and coordination failure modes. Multi-agent is an optimization for proven decomposition problems, not a starting architecture.
03 · ARCHITECTURE
Reference architecture
The orchestrator plans and delegates; workers are narrow specialists; shared state is the single source of truth; and the policy gate plus human approval sit between the agents and anything irreversible.
04 · COMPONENTS
What each piece does
| Component | Responsibility | Build/buy notes |
|---|---|---|
| Orchestrator / planner | Decomposes the request, assigns subtasks, merges results, handles failures | Often the smartest model you can afford; planning quality dominates outcomes |
| Worker agents | Narrow specialists (researcher, analyst, builder, critic) with constrained tool sets | Keep prompts small and role-specific; workers should be boring and predictable |
| Shared state & memory | Task graph, artifacts, conversation history — visible to all agents | The difference between "agents" and "a group chat"; design the schema first |
| Tool & API layer | Allowlisted tools with scoped credentials, executed in sandboxes | Tools are the real attack surface — see Security |
| Policy gate | Evaluates cost, risk, and compliance rules before high-impact actions | Deterministic rules first; LLM judgment only where rules can't reach |
| Human approval | Explicit sign-off for irreversible or externally visible actions | Design the approval UX as carefully as the agent — approvers need context, not just a button |
| Eval harness | Task suite with graded outcomes, replay, and regression tracking | Task-based evals (did it complete the workflow?) beat conversation evals |
05 · FLOW
How work moves through the system
06 · TRADEOFFS
Decisions with real costs
| Decision | Option A | Option B | Our default |
|---|---|---|---|
| Topology | Central orchestrator — simpler to reason about, single bottleneck | Decentralized / swarm — resilient, much harder to debug | Central orchestrator; decentralize only with proven need |
| Worker models | One strong model everywhere — simple, expensive | Small models for workers, strong for planner/critic — cheaper, more prompt work | Tiered: strong planner + critic, efficient workers |
| Communication | Natural-language handoffs — flexible, lossy | Structured artifacts (JSON schemas) — rigid, reliable | Structured artifacts; language only for summaries |
| Autonomy | Full autonomy — fast, risky | Human-in-the-loop everywhere — safe, slow | Risk-tiered: autonomous for reversible, approval for irreversible |
07 · SECURITY
The tool layer is the attack surface
- Least-privilege tools: each worker gets only the tools its role needs, with scoped credentials (read-only DB roles, sandboxed shells).
- Prompt-injection containment: tool outputs are untrusted data; workers must not treat them as instructions. The critic re-validates any instruction-like content from tools.
- Credential hygiene: agents never see raw secrets — tools broker authenticated calls server-side.
- Blast radius: destructive tools (delete, deploy, send) require policy-gate pass and human approval; there is no "agent root."
08 · GOVERNANCE
Accountability for autonomous work
- Full audit trail: every plan, delegation, tool call, and approval is logged with timestamps and attributed to an agent and a human owner.
- Approval policies as code: which actions need human sign-off is versioned policy, not tribal knowledge.
- Kill switches: per-run and global stops, plus automatic termination on budget breach or loop detection.
- Change control: worker prompts, tool allowlists, and policies are versioned; evals gate every change.
09 · COST
Where the money goes
- Tokens multiply: N agents × their context × retries. A 5-agent workflow can cost 10–20× a single-agent call for the same task — budget per run, not per call.
- Retries dominate: the critic loop is where cost accrues; cap iterations and make "escalate to human" cheaper than "try again."
- Model tiering is the biggest lever: planner and critic on frontier models, workers on efficient ones.
- Observability isn't free — full traces at high volume need a retention and sampling strategy.
10 · IMPLEMENTATION
A phased path that de-risks
Engagement tie-in: our Agentic AI Workforce offering builds these systems as production engineering, and the Agentic AI Readiness Assessment tells you whether your use cases are agent-ready.
11 · RELATED
Keep exploring
Enterprise RAG
Grounded retrieval is the knowledge layer most agent systems need underneath.
Read pattern →Agentic AI Workforce
Our flagship offering: production agent systems, deployed and operated.
Explore offering →Evaluating AI Agents
Our engineering brief on what to measure before you trust an agent in production.
Read article →Start here
Talk to an Architect
Bring your hardest AI, data, or modernization problem. We'll tell you plainly whether we can help — and what it takes.
FAQ
Questions we hear
When the work decomposes into genuinely different skill profiles (research vs. code vs. verification) and a single context window can't hold the plan, the tools, and the working state. If one agent with good tools passes your evals, ship that.
Budgets per task and per run, kill switches on loops, and a critic agent that can terminate unproductive branches. Most runaway cost comes from retry loops with no termination condition — make termination a first-class design decision.
Debuggability. You need full traces: which agent did what, which tool calls were made, what the intermediate states were. Without replayable traces, every failure is a mystery.