Cache survives the fork
Our planning mechanism fans out a full plan to every coder agent. The worry was cost: each coder would re-read the entire plan. A probe shows they don't — 97% cache hit within a model. Cross-model cache is 0%, but that's bounded, not catastrophic. And full-context coders are dramatically more accurate.
The fork
agntro's planning phase (call it A8) runs a variation ladder: multiple planner drafts, a validator that diffs new findings against the running set, an Arbiter that consolidates everything into one final plan. Then comes the fork — the Arbiter's plan is given to each coder agent, along with its specific task.
Each coder receives [system prefix + full Arbiter plan + its task].
Full scope, not a slim slice. The design bet: coders produce better code when
they see the whole plan — the helper functions, the integration points, the
contracts — not just their narrow assignment.
The worry was cost. If five coders each re-read a 4000-token plan from scratch, that's 20,000 input tokens of plan reading. Does prompt caching make this cheap, or does the fork multiply the cost?
Two assumptions under test
The fork rests on two assumptions, and we tested both:
- A8 quality: coders with full-plan context produce materially better code than coders with a slim task-only slice.
- B2 cache economics: the fork is cache-cheap — coders share the plan prefix via prompt caching, so you pay for the plan once, not N times.
The cache result
97% cache hit on every coder lane — including the first one.
With planner + all coders on the same model and a byte-stable
[system + Arbiter plan] prefix, only the per-task tail is fresh.
"We pay for reading + reasoning the plan once" holds for a same-model fork.
The constraint is real and worth stating plainly: prompt caches are keyed on
(model, prefix). Within a single model, the fork is cache-cheap.
Across models, cache reuse is exactly 0% — confirmed by a
probe where planner and coders ran on different models.
But that cross-model cost is bounded, not catastrophic. A mixed-model pipeline doesn't re-read the full conversation at each handoff — it passes a distilled artifact (the task-completion summary). So B2's "no cross-model cache" costs re-reading the summary, not re-reading the entire context. Mixed-model pipelines remain viable; they budget for the summary re-read.
The quality result
The cache numbers would be irrelevant if full-plan context didn't actually help.
We tested the quality assumption against ground truth — real litellm source
files, verified via the codebase graph. Each coder's output was checked against
the actual interfaces in types/utils.py,
cost_calculator.py, and complexity_router.py.
The result is decisive, not close:
Full-context coders matched the real codebase. They used
actual field names (litellm.model_cost[model]["cache_read_input_token_cost"]),
extended existing models rather than rewriting them, and got the class
hierarchies right.
Slim-slice coders hallucinated entire interfaces. They
invented self.cache_hit (no such attribute), a
cost_cache_hit() function with a fake
DEFAULT_CACHE_HIT_MULTIPLIER, and rewrote
class Usage(BaseModel) with wrong field names and the wrong base
class (real: Usage(SafeAttributeModel, CompletionUsage)).
The slim coder also burned 3050 reasoning tokens re-deriving context it didn't have — and still got it wrong. The full-context coder used 1039 reasoning tokens and got it right. The slim slice isn't just lower quality; it's more expensive in the dimension that matters for reasoning models.
What this means for the harness
- Give every coder the full plan. The quality delta is too large to ignore, and the cache makes it affordable within a model. Slim slices fabricate interfaces — that's a correctness problem, not just a style one.
- Keep planner and coders on the same model when possible. 97% cache hit vs 0% is a real cost lever. When you must mix models, budget for the summary re-read, not the full-context re-read.
- Cross-model cache is a constraint, not a blocker. Mixed-model pipelines are viable because handoffs pass distilled summaries. The cost is bounded and known.
What we didn't prove
The cache probe used a byte-stable prefix — the system prompt and Arbiter plan were identical at the byte level across all coder lanes. In practice, minor prompt variations (a dynamic timestamp, a per-task system tag) can break cache contiguity. The 97% is an upper bound for a clean prefix; real-world numbers will be lower if the prefix isn't disciplined. The harness now uses content-hashed prefix segments to keep the cache honest.