The Distributed-Systems Toolkit AI Agents Inherit
You added a second AI coding agent, then a third. Within a week you had the failure mode nobody warns you about: two sessions editing the same file at the same time, one agent overwriting another's finished work, a handoff that evaporated when a session died mid-task. The models are individually fine. The system around them has no coordination discipline — and the fix is not a smarter model. It is a toolkit that distributed-systems engineers assembled between 1987 and 2016, applied exactly once, correctly, to the swarm.
This post expands section 2 of the Koinessa white paper — the argument, in brief, that agent systems inherit every classical failure mode of distributed systems, and that the classical controls already exist. For each mechanism, we cover what the original work actually established, then how Koinessa applies it to multi-agent work. This is part of an ongoing series; the series introduction frames the verification problem this coordination layer feeds into.
Partial failure is the baseline, not the exception
The white paper's stance is blunt: treat every agent as a worker that can be slow, stale, duplicated, interrupted, or wrong. A session dies after modifying files but before reporting. A tool call times out after succeeding. Two workers genuinely believe they own the same resource. None of these are model errors — they are the ordinary physics of concurrent processes, and no amount of model intelligence substitutes for structural checks. Six mechanisms carry the load: leases, fencing tokens, idempotency keys, transactional outboxes, sagas, and reconciliation loops.
Leases: authority that expires instead of deadlocking
Cary Gray and David Cheriton introduced leases at SOSP 1989 in "Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency". Their problem was distributed file caching; their insight generalizes: grant authority for a bounded time, and a crashed holder releases that authority by doing nothing. A lock held by a dead process is a deadlock. A lease held by a dead process is a short wait.
Applied to agents: a work item on the board is held under a lease with heartbeats for liveness. If the session stalls — context compaction gone wrong, a crashed CLI, a laptop that closed — the lease expires and the item returns to the board for another worker. Nobody files a ticket to pry a lock off a dead process.
Fencing tokens: stale authority must never become current authority
Martin Kleppmann's 2016 "How to do distributed locking" — written as a critique of Redis's Redlock algorithm — made fencing tokens the standard companion to leases. The fix he describes is simple: the lock service issues a strictly increasing token with every acquisition, and the protected resource refuses any write whose token is lower than the highest it has already seen. Leases alone are not enough, because a paused process (his example is a stop-the-world GC pause, but a stalled agent session qualifies identically) can outlive its own lease without knowing it.
The white paper spells out the canonical agent-swarm accident once, because it is the one that silently corrupts work. Agent A takes lease number 41 on a work item and stalls. The lease expires; Agent B takes lease 42 and finishes the work correctly. Agent A eventually wakes and attempts to write its stale result. Without fencing, A silently overwrites B. With fencing, the protected resource remembers that 42 is the newest valid token and rejects 41. In Koinessa this is live machinery: cooperative file claims plus fenced work-item leases mean an agent that wakes up late discovers its authority lapsed at write time, not in a merge conflict three commits later.
Idempotency keys: retries that cannot double-execute
A tool call that times out after succeeding is the nastiest ambiguity in agentic work — did the charge post, the message send, the migration apply? Stripe's idempotent requests design is the reference answer: the client attaches a unique key to a request, and the server saves the resulting status code and body of the first request made with that key — whether it succeeded or failed, including 500 errors. Subsequent requests with the same key return the same stored result instead of executing again. Keys are up to 255 characters, Stripe suggests V4 UUIDs, and keys are pruned once they are at least 24 hours old.
For agent actions the rule becomes: anything consequential carries a durable idempotency key generated before the first attempt. Retrying after a timeout is then safe by construction — the system replays the recorded outcome rather than performing the action twice. This is what turns "the session died; rerun it" from a coin flip into a procedure.
The transactional outbox: durable intent before execution
Chris Richardson's transactional outbox pattern addresses a subtler split: a service must update its database and emit a message, and doing one without the other (crash after commit, before send) leaves state and notification permanently diverged. The pattern writes the outgoing message into an outbox table inside the same local transaction as the business-entity update; a separate relay then publishes from the outbox. Messages are guaranteed to be sent if and only if the transaction commits — and, as Richardson notes, consumers still need idempotency, because the relay may deliver more than once. The patterns compose; they do not substitute for each other.
For agents, the mapping is direct: record the intent — the file claim, the work-item state transition, the evidence write — in the same transaction as the state change it describes. A session that dies after committing but before announcing has not lost the handoff; the relay carries it. Intent is durable before execution begins.
Sagas: compensation for work that crosses irreversible boundaries
Hector Garcia-Molina and Kenneth Salem introduced sagas at SIGMOD 1987 in "Sagas". Their observation: long-lived transactions cannot hold locks for their duration without strangling concurrency, so decompose them into a sequence of local transactions, each paired with a compensating action. If a step fails, run the compensations in reverse order rather than aborting a global lock or leaving a half-applied state.
Long agent tasks — a multi-story requirements run, a migration touching dozens of files — are exactly long-lived transactions. Each completed story is durable on its own; a failure partway through triggers compensation of the steps that cannot stand alone, instead of stranding the repository in a state no session remembers creating.
Controllers: declare desired state and reconcile
Kubernetes replaced "run this and hope" with the controller loop: declare desired state, observe actual state, act to reduce the difference, repeat. The white paper applies the same shape to agentic work. A requirements document declares outcomes; work items bind obligations; verification profiles declare required checks; evidence schemas declare the proof that must exist. The controller then compares observed state against declared requirements — coverage exists or it does not; the artifact matches its schema or it does not — instead of asking the agent whether it feels finished. Vague autonomy becomes measurable convergence, which is also the premise behind the evidence-gated development loop elsewhere in this series.
Reconciliation also supplies the missing regulator for agent throughput: backpressure. When a downstream system slows or fails, controllers reduce work, queue it, or stop — whereas naive automation converts a small outage into a flood of retries and duplicate actions. Once every operation carries durable identity from intent to outcome, reliability becomes computable by agent, model, task class, and repository — and model selection stops being brand preference and becomes an evidence question.
What Koinessa runs today
These mechanisms are live, not aspirational. Session registration, cooperative file claims, and fenced work-item leases let interactive workers from six vendors — Kimi, Claude, Grok, Codex, local Ollama, and Ollama Cloud — share one board. A seventh vendor, Devin, is registered as the sixth provider descriptor and remains disabled behind a committed guard. The session registry holds 209 registered sessions, 208 of them since August 1.
The white paper is equally specific about measurement limits, and they matter for credibility. A provider label is captured on a minority of session rows — 8 of 209 at this snapshot — so per-vendor attribution rests on commit trailers, which record 18 distinct worker identities since July 1 and are not universal. The 2026-08-23 git snapshot contains 100 mainline commits from August 19 through August 23, 691 from August 5 through August 23, and a peak of 84 on August 13. Two human operators on separate LAN workstations can — and do — direct independent swarms against the same repository through the same coordination plane. That is concurrent-operator tenancy on one shared brain, and it is operating evidence from one system, not a controlled benchmark.
The 2026-08-12 receipt: a multi-vendor swarm on one board
One repository receipt shows the toolkit under load. On 2026-08-12, a single orchestrating session scanned the then-266-card work board, ran two explorer agents in parallel to map two code seams, had a local 27B model draft the requirements document from their reports, ran two coder agents in parallel on disjoint workstreams implementing four stories, and then independently re-ran the full verification suite — 11,271 tests green — before the work was accepted. No session's context held the project; the Brain did.
External measurement keeps this honest about what happens without coordination: a 2026 study of agentic pull requests found that only 0.5% of co-active pairs involved different agents — that is 0.5% of co-active pairs, not of pull requests — and in a replay sample, textual Git conflicts were 41.7% versus 19.8% for the cross-agent population; those are textual conflicts, not semantic ones (arXiv). Collisions between uncoordinated workers are the norm. The mechanisms above are how one system makes them the exception.
Takeaway
Duplicate work, overwritten changes, and lost handoffs are not AI problems — they are the classical distributed-systems failures, and they have classical fixes. Leases bound authority in time. Fencing tokens ensure stale authority never becomes current authority. Idempotency keys make retries safe. The outbox makes intent durable before execution. Sagas make long tasks recoverable through compensation. Controllers reconcile declared state against observed state and apply backpressure when downstream slows. Koinessa runs all six as live coordination machinery on customer-controlled compute, so a parallel swarm produces a verified green pipeline instead of a merge conflict. The full argument — with the snapshot data, the capability labels, and every citation above — is in the Koinessa white paper. For the wider frame on why these lessons were chosen, see seven lessons from fifty years of systems.