Why Your Multi-Agent Pipeline Is Slow and Expensive (And How to Fix It Systematically)

We built a multi-agent pipeline to automate a complex, document-intensive enterprise workflow. The initial system ran sequentially through a chain of specialized agent roles, each handling a distinct task in the process, using a single frontier model throughout. It worked. It was also slow and expensive, and we didn’t know why.

Rather than optimize blindly, we profiled first. The results were not where we expected: 67% of total latency came from a small minority of the agent roles, and the primary bottleneck was not the model — it was sequential chaining: independent work was being processed one step at a time instead of concurrently. Identifying and parallelizing the roles with no inter-dependency reduced end-to-end runtime by roughly 40%.*

The second experiment compared model tiers (frontier, mid-tier, lightweight) across each agent role independently, measuring accuracy, latency, and cost per role. The finding cuts against the default assumption: the roles that appeared most cognitively demanding required frontier models, but the accuracy gap was smaller than expected. The roles where model downgrade failed were the ones responsible for precise structured extraction — tasks where errors propagate silently downstream and surface only at the output. Swapping to mid-tier models on roles that tolerated it reduced per-run cost by ~40%, with an estimated <2% accuracy impact at the role level and <1% at the pipeline output level.*

The third experiment compared orchestration harnesses — evaluating how the choice of agentic framework affects runtime overhead, observability, and the ease of implementing the parallelization and model-swap changes described above. Framework choice turned out to matter more than expected for operational concerns: debugging multi-agent failures, tracing costs per agent, and modifying execution flow without rewriting pipeline logic.

Operating in a regulated enterprise environment added one constraint worth naming: every agent decision needs to be auditable. This ruled out certain optimization shortcuts that would have been acceptable in other contexts and shaped how we defined “accurate enough” per role.

This talk covers the profiling methodology, the per-role model comparison framework, the parallelization decisions, the harness comparison, and the operational lessons — including what we’d instrument from day one if we rebuilt the system today.

*Cost and accuracy figures for the parallelization and model-tier experiments are preliminary; the formal evaluation is in progress and will be updated with measured results before the talk.