Why multi-agent AI systems fail in production

A single agent with a good prompt is reliable enough to be useful. Chain five of them together and the thing falls over. This is the most common experience people have with agentic AI, and it is almost never a model problem.

The arithmetic nobody runs before building

Suppose each step in your agent chain is 95% reliable. That sounds strong. Run five steps in sequence and the chance all five are correct is 0.95 to the fifth power — about 77%. Ten steps takes you to roughly 60%. Twenty steps and you are at a coin flip.

Nothing has gone wrong with any individual agent. The failure is structural: independent error rates multiply. This is why a demo that works beautifully on stage produces garbage on the fortieth run, and why "the model got worse" is usually the wrong diagnosis.

Worse, errors in a delegation chain are not independent in the helpful direction. A manager agent that misreads the goal hands a subtly wrong brief to three workers, and all three produce confidently wrong output that is internally consistent. The system looks healthy right up until a human reads the result.

Four failure modes worth naming

1. Context dilution

Each hand-off summarises. Summaries lose specifics — the constraint, the exception, the client's actual name. By step four the agent is working from a paraphrase of a paraphrase, and it has no way to know what was dropped.

2. Confident drift

Agents rarely stop and say they are unsure. They produce something plausible. In a chain, plausible-but-wrong is far more damaging than an obvious failure, because the next agent treats it as ground truth and builds on it.

3. Runaway loops

Two agents negotiating a task can ping-pong indefinitely. Without a hard stop this is not just a quality problem, it is a billing one — the meter runs while the loop spins, often overnight, often discovered by the invoice.

4. No accountable observer

The deepest one. In most deployments nobody is watching the output between the agent producing it and the customer receiving it. The system is autonomous in the literal sense: unsupervised.

What actually fixes it

The fixes are boring, structural and mostly not about models.

• Shorten the chain. Two well-scoped agents beat six chatty ones. Every hand-off you remove removes a multiplication.
• Give every task its goal ancestry, not a summary. An agent that can see why it is doing something catches briefs that drifted.
• Put a human gate before anything leaves the building. Not on every internal step — on the boundary where a mistake reaches a customer.
• Cap spend per agent with a hard stop. A budget ceiling turns a runaway loop from a financial event into a paused agent and an alert.
• Log every tool call immutably. If you cannot answer "why did it do that", you cannot fix it, and you certainly cannot explain it to a client.
• Verify adversarially. Have something check the output against the original goal rather than against the previous step.

Why this shapes how we build

We run departments of two to three agents rather than sprawling crews, because short chains fail less. Anything that would leave your business waits in an approval queue. Every agent carries a hard monthly spending ceiling and stops when it hits it. Every action is logged and traceable. None of that is exotic — it is the structural response to error compounding, applied by default rather than bolted on after the first bad week.

More detail on the controls is on our security page, and the approach is compared against the main frameworks in our alternatives guide.

Keep reading

• Every agent platform we get compared to, side by side