New: FixBugs for VS Code is now available. Install from Marketplace 

ai agents

Multi-agent coordination is a harness problem

Three experiments on multi-agent systems point the same way. Agent coordination is a harness problem.

Kirti Rathore··11 min read

"Over time, delightful innovation becomes another basic need."

That's the Kano model, and coding agents went through the cycle in about two years.

Codex and Claude Code have over 50 million users between them. Nobody is delighted that an agent can automatically generate tests for human-written code anymore.

Kano model: satisfaction plotted against how fully a capability is implemented. Delighters rise steeply once implemented, performance needs rise linearly, basic needs only remove dissatisfaction. A dashed arrow shows delighters drifting down to basic needs over time.
Satisfaction against implementation. What delights today is a basic need tomorrow.

As coding agents became reliable, expectations for complexity, quality and speed grew as well.

We've moved from a single agent solving one problem to a team of agents solving more complex, less defined tasks.

However, agent co-ordination has proven to be difficult, time-consuming and error prone.

The demand for higher quality work, faster, have led to the emergence of new multiagent co-ordination patterns.

Here are two that have found wide adoption:

  • Orchestrator. Agents have excelled at tool use for some time. As long as one agent can treat another as a tool call, with a prompt in and a response or artifact out, they work together efficiently.

  • Parallel agents. Work is mapped across a set of agents, then a reducer agent consolidates the results and proposes the next action. Interference between agents is minimized.

In both these patterns, agents never speak to each other.

A recent Anthropic blog explored a third pattern: agents that talk to each other directly.

A swarm co-ordinates on a task through a shared communication channel, without an orchestrator or a contract.

tl;dr

  • Swarms beat parallel agents on search tasks. A 45-agent swarm, free to pick where it looked, found 266 vulnerabilities. The same agents with the repository files split between the agents found 21.

  • Errors grow with the size of the swarm. A swarm was told to build a game. As it grew from 10 to 80 agents, the share of pull requests that merged fell sharply. The model used did not make a difference.

  • Groups lose information individual agents possess. When the answer depended on a fact only one agent held, and the agents were given a shared forum to post to, four-agent groups found it in 17–36% of episodes. One agent given every fact scored near 100%.

  • In production, most failures are specification and verification. Across 1,600+ traces from 7 frameworks, 41.77% of failures were specification issues and 21.30% were verification failures.

reads scale, writes do not

The experiments pointed communicating swarms at two very different workloads.

The first was vulnerability discovery.

45 agents, 15 open source repositories, one VM each, a shared forum, peer review, and a separate arbiter agent to accept findings.

The swarm found 266 vulnerabilities across 27M sampled output tokens. Independent parallel agents on the same task found 21 across 6.5M tokens. Discoveries increased linearly with the token budget. Four times the tokens bought twelve times the findings.

Being able to move attention to wherever vulnerabilities looked cheapest to mine worked better than splitting the work into discrete filesets.

45 agents with a shared forum, each picking where to look, produce independent findings and reach 266 vulnerabilities against 21 for parallel agents.
A read task. A miss by one agent is caught by another agent, and this works well.

Then they pointed the same design at a write task. The second workload was building a game: 10 to 80 agents, twelve hours, one shared repository. This one went badly.

10 to 80 agents with a shared forum all write to one shared repository, and the merge rate collapses, from 52 to 9 percent for the worst model.

Pull requests against one repository turned into merge conflicts, contention and misaligned work.

Dumbbell chart showing the share of pull requests merged at 10 agents versus 80 agents for five models. Every model falls: 52 to 9 percent, 68 to 18, 93 to 66, 95 to 61, and 88 to 77 percent.
Every model falls. Some fall further.

using just one model leads to a thundering herd

Agents using the same underlying model follow the very similar paths in the problem space.

In one game-building run, 18 of 30 agents independently created a git branch with the same name: mvp-game-loop.

Asked to build something impressive, over half chose either a ray tracer or a self-hosting compiler.

This behavior, with massive agents swarms can quickly lead to an unintended DDOS attack on the system being operated upon,

When the agents were given a finite-bandwidth job queue to pull from, they independently converged on aggressive polling.

One run produced 2.4 million requests and overloaded the queue. The acceptance rate turned out to be 20,500 requests for one successful admission.

liars and lone dissenters

The second half of this class is epistemic. In a deception task, a listener made 10–15 decisions using reports from four scouts, one of which lied at a fixed rate. Newer models got substantially better at spotting the liar, though none matched the oracle policy that simply discards it once contradictions expose it.

Then the same models got the reverse problem. Four agents each hold part of a picture. The shared evidence points at the wrong answer; the decisive fact sits with one agent. After discussion, they vote.

Bar chart of the share of episodes where a four-agent group reaches the correct answer: 17, 18, 18, 36 and 85 percent across five models, against a dashed reference line at about 98 percent for a single agent given all the facts.
Four agents with split facts do far worse than one agent with all of them.
A trust dial from skeptical to receptive. The deception task pulls it left, discount the lying scout; the hidden-profile task pulls it right, believe the one agent with the decisive fact.
The two tasks pull the same dial in opposite directions.

No single trust setting solves both. Deception detection rewards skepticism; hidden facts reward taking a lone dissenter seriously. Turning one dial moves both tasks, in opposite directions.

Twenty identical agents are one fault domain.

where real systems break

Designed experiments tell you what can happen. Berkeley's paper Why Do Multi-Agent LLM Systems Fail? (Cemri et al.) looked at what does happen.

They annotated 1,600+ agent execution traces across seven real frameworks and built a taxonomy of 14 failure modes.

Stacked bar showing failure share across three categories: specification issues 41.77 percent, inter-agent misalignment 36.94 percent, task verification 21.30 percent, with the leading failure modes listed under each.
Read the leaf nodes as a list of things you can fix without touching a model.

Step repetition is a state problem. Missing termination conditions is a spec problem. Never asking for clarification is a protocol problem — the agent had the option to ask and didn't take it. None of these get solved by swapping in a better model.

design rules

  • Decompose before you parallelise. Decide what is a read task and what is a write task before you decide how many agents.
  • Measure merged work, not activity. PRs opened, messages sent and tokens burned all rise while throughput falls.
  • Diversify deliberately. Vary context, scaffolding or model, or you get correlated errors dressed up as agreement.
  • Keep provenance on every claim. An agent should be able to tell repetition from corroboration.
  • Give conflicts a protocol and an arbiter. Destructive actions against a peer need stronger authorisation than ordinary work, and a path to a human.

None of these are validated mitigations. They're what the results imply, and they're cheap to test.

That is how we build agents that touch your codebase: read-heavy work with a verification step.

reading list