Fan-Out, Fan-In: How an Orchestrator Actually Delegates
An orchestrator delegates well only when each brief is a contract that code checks before dispatch, and each result comes back as a summary and a pointer.
Written as at 19 August 2026.
agentic-aimulti-agent-systemsorchestrationtask-ledger
“Research prompt caching” is not a brief. Hand it to three workers and you will get three overlapping essays, one gap nobody covered, and a bill for all of it.
Anthropic found the same thing building its multi-agent research system. Its lead agent at first gave subagents short, vague instructions, and they duplicated each other’s work, left gaps or missed information. The fix was to give every subagent an objective, an output format, guidance on tools and sources, and clear task boundaries.
This piece is about that handover: how an orchestrator plans, what a brief must carry, what to check before any worker starts, and what should come back. The example is Poly, the studio I am building, where an orchestrator builds Lesson 3.2 on prompt caching: a script, three diagrams and a five-question quiz.
The plan is a task graph, and it lives in a ledger
The orchestrator’s first move is one planning call that writes a task graph. For the lesson, research (T1), diagrams (T2) and the quiz (T3) are independent, so they run in parallel. The script (T4) needs the research. Quality review (T5) needs everything. In short: T1, T2 and T3 together, then T4, then T5.
That graph goes into a durable task ledger, not only into the orchestrator’s context window. Each row carries an owner, dependencies, acceptance criteria, a budget, an idempotency key and a status. If the orchestrator’s container restarts halfway through, a new one reads the ledger and resumes from the last completed task instead of starting again and paying twice.
The ledger also makes fan-out safe to size. A burst of parallel workers can exceed a provider’s tokens-per-minute limit at once, so the dispatcher needs a concurrency limit sized to that rate. Retries need exponential backoff with jitter, as Marc Brooker describes in the Amazon Builders’ Library, so failed calls do not all retry at the same moment.
A brief is a contract with six fields
A brief is the only context a worker gets. It starts with a fresh window holding its own system prompt, its tools and the brief, with none of the orchestrator’s history. So the brief has to be complete.
- Objective. What to find or make, and what is out of scope. “How prompt caching works, its pricing mechanics, two real examples” is an objective. “Research caching” is a topic.
- Output contract. A named schema, a maximum summary length and where the full artifact goes.
- Tools. The exact list this task may call, which the tool gateway enforces.
- Budget. Maximum turns and maximum spend for this task, drawn from the run’s budget.
- Deadline. When the orchestrator stops waiting and re-plans.
- Idempotency key. A key such as l3.2-T1-r0, so a retried task cannot write twice, provided every write tool honours it.
Trace context travels with it, so the worker’s spans join the run’s trace. In the illustrative Poly build, the research brief looks like this:
{ "task_id": "T1", "agent": "researcher",
"objective": "How prompt caching works, pricing mechanics, 2 real examples",
"output_contract": { "schema": "ResearchResult@v2", "summary_max_tokens": 900,
"artifact": "s3://poly/l3.2/research.md" },
"limits": { "max_turns": 8, "max_usd": 0.25, "deadline_s": 120 },
"tools": ["web_search", "fetch", "artifact.put"],
"trace": { "trace_id": "4bf9…a1", "parent_span": "orch.plan" },
"idempotency_key": "l3.2-T1-r0" }
Two of my constructs help here. A Context Contract is a declared, versioned and testable specification of what context an agent purpose may receive (see Context Is Not a Prompt). The researcher role has one; each brief is an instance that must fit inside it. An Authority Envelope is the explicit bounds on what an agent may do: its tools, resources, actions, values and duration. The tool list and budget in a brief are that envelope for one task, and the orchestrator can narrow it but never widen it (see Delegation Is Not Authority).
Check every brief in code before it is dispatched
The orchestrator is a model, and models write incomplete briefs. So a small piece of deterministic code sits between the plan and the workers and checks every brief against its schema. Figure 1 shows the three outcomes.
The research brief passes all five checks and is dispatched. The diagrams brief names no output schema, so it is returned to the orchestrator with the reason, and the orchestrator re-plans it on its next turn. The quiz brief asks for $0.40 against an illustrative per-task cap of $0.25. Raising a cap is a decision beyond the orchestrator’s authority, so the brief is escalated to a person. Each outcome writes a row to the ledger, which is how you later explain why T3 started late.
None of these checks needs judgement. They are facts code can test, which is exactly why the model should not be the one testing them.
Fan-in returns conclusions, not raw material
Workers read the raw material in their own windows. In the illustrative build, about 50,000 tokens of pages, diagrams and syllabus text arrive across the three workers. None of it should reach the orchestrator.
Each worker writes its full output to the artifact store and returns a small envelope instead: status, a summary, the artifact URI, source count, confidence, usage and a trust tag such as “derived from untrusted web”. The envelope is checked against its schema before the orchestrator reads it. In the Poly build, the orchestrator’s window grows by about 1,400 tokens rather than 50,000, and the worker windows are thrown away.
Dependencies then unblock through the ledger. T1 is done, so T4 starts, and the scriptwriter’s brief carries the research URI, not the research text. Failures come back typed as well: “status: failed, reason: tool_unavailable” lets the orchestrator re-plan or fall back in code, without parsing an apology. You can step through the fan-out and worker-failure scenarios in the agent simulator.
Contracts make delegation checkable. They do not make a bad plan good. If the orchestrator splits the work along the wrong lines, every brief can pass and the lesson still fails review. A dispatch gate also cannot tell whether an objective is the right one, only whether one exists.
Who does what
Executives: ask whether every delegated task has a budget and an owner in a durable record. If the answer is “the orchestrator keeps track”, a restart will cost you a full rerun.
Architects: define the brief and envelope schemas once, version them, and make the dispatch gate and ledger shared platform components.
Engineers: reject briefs without a schema, budget or idempotency key in code, test that a retry with the same key writes nothing new, and propagate the trace ID into every worker.
This article is part of The Agentic AI Architect, a series from the author of The Contextual Agentic Enterprise. Each piece matches a module of his course in development, Agentic AI Architect Pro.
References
- The Contextual Agentic Enterprise.
- Anthropic (13 June 2025) "How we built our multi-agent research system". https://www.anthropic.com/engineering/multi-agent-research-system
- Brooker, M. "Timeouts, retries, and backoff with jitter". Amazon Builders' Library. https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
Related
Start With One Agent: Choosing the Simplest Pattern That Works
Multi-agent designs are not a maturity level. Each rung from a single call to orchestrator-workers adds cost and failure points, so climb only when the work branches.
The Ten Layers Behind Every Multi-Agent System
A multi-agent demo has three boxes. Production has ten layers, each owning one job and failing in one familiar way. Most incidents start in a layer nobody owned.
Delegation Is Not Authority: Governing Multi-Agent Handoffs
When one agent hands work to another, authority should narrow, never widen. A delegated task must carry its origin and its limits, or it becomes a way to launder a prohibition.