When smart agents failed each other
We expected the models to break. Not for the reason we feared. The surprising finding: breakdowns came from organization and messages, not from model smarts. A UC Berkeley study placed 36.9% of multi-agent failures on inter-agent miscommunication. We saw the same thing, and I can show you the moment it stopped being theory and became obvious.
The experiment we ran, step by step
We built a pipeline with a signed, formal protocol on paper. Nine message intents. HMAC-signed canonical JSON. Idempotency keys with 90-minute deduplication. A taxonomy that said validation/auth errors never retry and timeouts get bounded retries. It read beautifully. It looked safe.
Then we started running real work through the system. We turned on the agents but left some infrastructure unbuilt (message queues not yet implemented), and we never configured signing keys. We still wanted to see how far the agents could get under realistic pressure.
Result 1: duplicate execution. Same job arrived twice. One worker did it both times. Cost doubled. We watched logs: idempotency keys were present in the spec, absent in the running stack. The system retried blindly.
Result 2: lost thread context. A coordinator asked a worker to fetch a file and return a checksum. The worker sent back a result but the coordinator couldn’t link it to the original request because the thread metadata didn’t travel with the message. The file was marked “missing” and a human had to reconcile.
Result 3: runaway recursive chains. An agent, asked for clarification, delegated to another agent, which interpreted the delegation as new work and delegated again. We saw the chain double in five minutes and then hit a cost threshold. No cryptographic guard stopped it — only a manual cutoff outside the agent code did.
The turning point came when one hand-off actually contained verifiable proof: a byte size and checksum attached to the result. The coordinator accepted it immediately, verified the checksum matched the file, and advanced the workflow without human intervention. The job finished. That one concrete receipt unblocked the whole run.
What changed after that moment
We stopped assuming agents could “remember” prior messages. We started assuming cold start — every message must carry the context needed to act. We stopped trusting an unimplemented spec. We focused on what actually ran in production: file hand-offs, receipts, five practical rules, and a human in the loop for the parts that cryptography didn’t yet cover.
The five plain rules that kept us safe
These were not theoretical safeguards. They were the layer that actually ran and prevented the worst failures.
- No recursive delegation: agents were forbidden from delegating to a peer that could delegate again.
- Verify before delete: any deletion required checksum or byte-size confirmation.
- Hard file boundaries: agents could touch only listed file IDs; no wandering filesystem access.
- Deterministic checkpoints: progress points that could be replayed deterministically if needed.
- Full audit trail: every event recorded who sent what and when, with receipts for payloads.
Plus: a cost circuit breaker lived outside the agents’ code and a human stayed in the decision loop for exceptional cases.
The rule that fell out — concrete and actionable
Make every hand-off verifiable: attach a receipt (checksum or byte size) to the result and require it before the next step. Assume cold-start: do not rely on agent memory. Put the necessary context in the message itself (thread IDs, correlation IDs, and the receipts). That single change converted ambiguous, lossy interactions into verifiable state transitions.
What we had designed but never actually ran
The protocol looked complete on paper: signed messages, HMAC, idempotency, dead-letter queues, a clearly defined error taxonomy. But the signing keys were unconfigured and several message queues were never built. So the elegant cryptographic layers stayed theoretical. In practice, agent-to-agent security rested on the five rules and a human — not on the spec’s cryptography.
A short history that matters
This maps to older agent work. Contract Net (1980) and speech-act styles framed delegation and bidding. Today there’s a split: one standard connects agents to tools; another connects agents to agents. The problems repeat — duplicate runs, lost messages, lost thread context, unauthorized actions, runaway chains, identity forgery, cost blowouts — seven failure modes we planned against and then observed.
One fair counter-example
Not everything failed. When we did manage to run the signed-path fully (keys configured, queues operational), the formal protocol worked as intended: idempotency avoided duplicates, signatures prevented easy forgery, bounded retries handled timeouts. The counterexample shows the specification is correct when implemented. But the honest behavior was: elegant layers are fragile if they remain only design artifacts. Running code mattered more than the plan.
How we know — evidence, not claim
Concrete traces backed this up. We have logs showing duplicate job IDs without deduplication, coordinator logs with missing thread correlation, audit entries where deletion was blocked until checksums matched, and a recorded run where a checksum receipt allowed an automated commit. The Berkeley study (arXiv:2503.13657) aligns with our numbers: a huge share of failures come from communication and organization, not model capability.
Practical rule, stated plainly
Every inter-agent hand-off must carry verifiable evidence of the work (checksum or byte size) and the minimal context needed to continue (thread/correlation IDs). Assume cold start. Design for receipts first; add signatures and queues later, but don’t ship without the receipts.
FAQ
Why assume cold start?
Because memory didn’t prove reliable in our runs. Agents restarted, queues dropped, and thread state vanished. Putting context in the message made each step self-contained and verifiable.
Are signatures and idempotency unnecessary?
No. They are necessary and correct in the spec. They become effective only when implemented. In our case the spec was cleaner than the running system; that gap mattered more than missing features in the models.
Is human oversight always required?
Not always, but until the missing engineering (queues, signing keys, replay protection) is in place, a human as a circuit-breaker and adjudicator prevents cost blowouts and unauthorized actions.
We stopped blaming the models. We started fixing messages and receipts. That change saved money, time, and a lot of hair.
Sources: internal pipeline experiments and the UC Berkeley MAST study (arXiv:2503.13657).
Leave a Reply