Context Economics: Cache Reads, Context Rot, Retrieval-First
Your cloud AI bill went up again this quarter, and the hard part to explain to yourself is why. The team is not shipping proportionally more. Hosted CI minutes, agent tokens, engineering-tool seats — the lines keep growing while output grows more slowly. If you have ever opened a provider dashboard and wondered what you actually paid for, the honest answer is usually: you paid to remind the model of things it already knew.
That is the claim the Koinessa white paper puts numbers on in its context-economics sections, and it is worth slowing down on, because the numbers are more specific — and more bounded — than the usual "AI is expensive" discourse. This post walks through the evidence exactly as the paper states it: what is measured, what is derived, and what each figure does not establish.
The bill is mostly re-read history
Start with Anthropic's own Claude Code cost documentation, which publishes a worked session example of how a coding-agent session spends tokens (code.claude.com). The white paper takes that worked example and does the arithmetic: 85.6% of input-side processing — uncached input, cache creation, and cache reads combined — is context being carried, and cache reads alone account for about 51% of it. Two qualifiers matter here, and the paper states both. First, the 85.6% is a derivation from the vendor's published example, not a statistic the vendor prints; the arithmetic is preserved in the paper's research digest. Second, it describes one worked example's input mix — it does not establish realized spend for any organization.
The variance picture is just as striking. Bai et al.'s 2026 study "How Do AI Agents Spend Your Money?" measured token consumption across eight models on OpenHands with SWE-bench Verified and found roughly a 1,000-times cross-run token difference and up to 30-times variance on the same task (arxiv.org). Same benchmark family, same task, wildly different meter readings. The boundary the paper attaches: this does not generalize to every agent workflow. But paired with the worked-example derivation, the direction is hard to argue with — in agentic coding, the dominant cost driver is context carriage, and it is highly sensitive to how the harness manages that context.
Twenty-two point seven billion tokens, itemized
Internal telemetry is where most vendors get vague. The white paper does the opposite, so here are its numbers with their boundaries intact.
Across 27 recorded days through 2026-08-16, the Koinessa operator's own workload consumed 22,696,191,284 total tokens, of which 20,918,082,610 — 92.17% — were cache reads, list-priced at $12,027.14; no later day had been ingested as of the 2026-08-23 snapshot. A separate 30-day window (2026-07-11 through 2026-08-09) shows the same shape: 18,284,233,434 total tokens, 17,088,709,895 of them cache reads — 93.46% — list-priced at $13,910.58. Cache reads are history being re-carried to the model, not new work being done.
The paper is explicit about what this is not: it is not a controlled productivity comparison, and it does not prove what a counterfactual workflow would have spent. What it does is identify where cost accumulates. Over nine dollars in ten of that list-priced window bought re-reading. That is the target any cost architecture should aim at, and it is why the paper's controlling KPI is verified work per dollar, attributed per work item — not raw token counts celebrated as throughput.
Long context is not cheap memory
If context carriage is the cost driver, the natural counter is "context windows are huge now, just carry everything." The cited research says otherwise, twice.
Modarressi et al.'s NoLiMa benchmark (ICML 2025) extends needle-in-a-haystack testing with needle-question pairs that have minimal lexical overlap, so models must infer latent associations rather than pattern-match (arxiv.org). Evaluating 13 models that claim at least 128K-token contexts, they found that at 32K tokens, 11 of 13 models dropped below 50% of their strong short-length baselines. Even GPT-4o, one of the top exceptions, fell from an almost-perfect 99.3% baseline to 69.7%. The white paper carries this as a benchmark-specific result — which is exactly how NoLiMa presents itself.
Chroma's "Context Rot" research (Hong, Troynikov, and Huber, 2025) generalizes the discomfort (trychroma.com). Across 18 models and deliberately simple controlled tasks, performance degrades as input length grows, in non-uniform and sometimes surprising ways — even on a trivial word-replication task. Their conclusion is blunt: whether relevant information is present in context is not all that matters; how it is presented matters more.
The third leg is operational. Anthropic's engineering writing on managed context notes that compacted messages remain in active context only in summarized form and are recoverable in full only when separately persisted (anthropic.com). The paper's reading is careful: the lossiness is an inference about unpersisted detail, not a claim that every durable log is destroyed. Together these results produce the architectural rule the paper draws: canonical state must live outside the model window. The window is working memory. It is expensive, it degrades with length, and its summarization is lossy unless you persist elsewhere. Treating it as the system of record is how you pay $12,027.14 list-priced to forget things.
Retrieval first, measured from both sides
The alternative to carriage is retrieval — fetch exactly what the task needs, when it needs it. The white paper brackets this choice with external studies on both sides.
In a read-only Python-repository question-answering study, Rafiei Oskooei et al. (2026) compared semantic retrieval against a deep agentic search approach: $0.32 per correct answer for semantic retrieval versus $0.74 for Deep — Deep about 2.3 times as expensive, semantic about 57% cheaper — with the largest token gap, 22.4 times, applying specifically to Qwen3-235B (arxiv.org). Boundaries: read-only QA on Python repositories, not whole-task performance, not every-model savings. Vogel et al.'s Codebase-Memory (2026), a structural code-graph approach, reported 83% versus 92% answer quality while using about ten times fewer tokens and 2.1 times fewer tool calls, with exploration retained as a fallback (arxiv.org). Boundary: it does not establish safe edits or that exploration can be dropped.
The counterposition is real and the paper states it plainly: Anthropic tried a retrieval index inside early Claude Code and removed it in favor of agentic search, citing staleness, privacy, and reliability. Those are architectural objections, not evidence that retrieval is wrong — and they are answerable architecturally. An index that is local-only, never leaves the machine, and is refreshed per commit addresses staleness and privacy directly; keeping grep as the declared fallback rather than a banned tool addresses reliability. Retrieval-first, as the paper frames it, is a cheap first hop, not a doctrine that the index replaces search. A sibling post walks through the retrieval strategy itself in more detail.
How Koinessa applies it
The design rule is one sentence: the cloud model has working memory, while Koinessa has project memory. Deterministic reducers and local inference assemble, compress, and format context before a frontier model ever sees it; every token assembled or reduced locally is a token the frontier provider does not bill. This is live, not roadmap.
Concretely, the code-intelligence index — Doxygen-ingested, 17,903 symbols for this project alone in the 2026-08-23 snapshot, last indexed the same day — answers "which file, which lines, what exact signature, which files depend on it" as one DuckDB query, in milliseconds, with a result small enough to paste into a context packet. Institutional knowledge gets the same treatment: PRDs, decisions, policies, and provenance records live as queryable tables and documents in the shared brain rather than being re-derived from chat history. And the rule is machine-enforced, not aspirational: a pre-tool hook refuses identifier lookups by text search across the Claude and Kimi harnesses and hands back the exact index query to run instead.
There is also a receipt from a harness Koinessa does not control. In a guarded sandbox probe on 2026-08-23 — one question, on one paid-tier session — the Devin CLI answered a symbol-location question through the brain's symbol index in four seconds: 44,496 prompt tokens (42,752 of them cache reads), 329 completion tokens, eleven steps, no text search. The same question routed through language-server tools cost 171,317 prompt tokens over sixteen steps — roughly four times the token cost. The paper's own qualifier applies: one question, one harness, one day; a receipt, not a benchmark. But it is the shape of the claim, observed in the wild.
This is the local-first cost advantage expressed at the context layer, and it composes with the rest of the architecture's components: the same brain that serves retrieval also holds the evidence and provenance records that make verified work per dollar measurable.
The takeaway
Rising AI bills are not primarily a model-pricing problem; they are a context-carriage problem. The measured picture — 85.6% of input-side processing derived from Anthropic's own worked example, 92.17% cache reads across 27 recorded days, NoLiMa's 11-of-13 collapse at 32K, and retrieval at $0.32 versus $0.74 per correct answer — points one way: stop paying frontier models to re-read history, persist canonical state outside the model window, and retrieve before you repeat. Every figure here comes with its boundary attached, and the full argument, with all citations and qualifiers, is in the Koinessa white paper.