Local-First AI Delivery: Cost Advantage by Architecture
The line item that only goes up
If you run an AI-assisted engineering org, you already know the shape of this invoice. Frontier API spend climbs as your agents get more capable and your sessions get longer. Hosted CI minutes scale with every pipeline you add. Per-seat tooling multiplies across every developer and every agent you put to work. None of these vendors is doing anything unreasonable — they charge for compute you consume. The problem is structural: you have built a delivery process whose marginal cost rises every time you use it more, and "use it more" is exactly what success looks like.
The question is not how to negotiate a better rate. It is whether the architecture you chose forces the bill. Section 6 of the Koinessa white paper argues that it does not have to — that local-first is an engineering architecture with a measurable cost structure, not an ideology, and that the savings come from decomposition and routing rather than from boycotting frontier models.
What the white paper section establishes
Section 6 makes three claims, all labelled live — running today, not roadmap.
First, local-first execution. The persistent shared memory at the center of the system is a local DuckDB file behind a local gateway. The verification engine runs on a local k3s cluster with a local Zot image registry: more than 69 deterministic verification jobs execute without purchasing a single cloud CI minute. Local models — Ollama-hosted, such as a 27B instruct/coding model running on the operator's own GPU — perform bounded work: context compilation, drafting, log reduction, failure classification, all at zero marginal provider cost. Frontier cloud models (Claude, Grok, Codex, Kimi, with Devin registered and disabled) remain in the loop, but as leased capacity for work that justifies that intelligence — not as the default path for everything.
Second, local compute does the context management. Deterministic reducers and local inference assemble, compress, and format context before a frontier model ever sees it. The operator's measured 30-day workload — 2026-07-11 through 2026-08-09 — shows why this matters: of 18,284,233,434 total tokens, 17,088,709,895 of them, 93.46%, were cache reads. History being re-carried, not new work being done. List-priced, that window estimates at $13,910.58.
Third, retrieval before repetition. Agents answer "which file, which lines, what exact signature" from a code-intelligence index — 17,903 symbols for this project alone in the 2026-08-23 snapshot — instead of burning frontier tokens on exploratory grep-and-read loops.
Each of these is a routing decision, and the routing rule is the heart of the section.
The escalation ladder: demonstrated capability, never brand
The design rule in section 6 is stated bluntly: do not build a local replacement for a frontier model. That project fails, and it fails expensively. Instead, decompose the workload until only a small fraction of it actually requires frontier intelligence, then route each fragment up an escalation ladder:
- Micro model — classification, extraction, formatting.
- Small local model — bounded drafting, log reduction, summarization.
- Larger local model — the 27B-class instruct/coding tier on owned hardware.
- Cheap cloud — hosted open-weight inference when a task outgrows the local GPU.
- Frontier — reserved for the narrow class of work that demonstrably fails everywhere below.
Escalation happens on demonstrated capability — measured failure on the current rung — never on brand, never on the assumption that the most expensive model is the safest default. This is the same routing discipline we described for AI context economics and retrieval strategy, applied one layer down: not just what context reaches the model, but which model is allowed to see the work at all.
What the external evidence actually says
This ladder is not a cost hack the white paper invented. Four external sources bracket it.
AgentFloor: a measured boundary of model necessity
Karmakar and Chatterjee's 2026 paper, "AgentFloor: How Far Up the Tool Use Ladder Can Small Open-Weight Models Go?" (HTML version), asks precisely the routing question the ladder encodes: which parts of an agent workflow truly require large frontier intelligence? AgentFloor is a deterministic 30-task benchmark organized as a six-tier capability ladder — instruction following, tool use, multi-step coordination, long-horizon planning — evaluated across 16 open-weight models from 0.27B to 32B parameters alongside GPT-5, over 16,542 scored runs.
The result is a clear boundary. Small and mid-sized open-weight models are already sufficient for much of the short-horizon, structured tool-use work that dominates real agent pipelines; in aggregate, the strongest open-weight model matched GPT-5 on the benchmark while being substantially cheaper and faster. The frontier advantage appears on long-horizon planning with sustained coordination — and even there, neither side reaches strong reliability. The white paper's own qualifier matters and stays attached: AgentFloor is a synthetic, read-only tool benchmark, and its results do not prove safe repository edits or production operations. It measures where the capability boundary sits, not whether crossing it is safe. That is exactly what an escalation ladder needs to know.
Gemma's E2B naming: what "small" actually means now
Google's Gemma documentation (model overview, Gemma 3n) clarifies the bottom rungs. E2B means effective parameters, not raw parameter count: Gemma 4 E2B advertises sub-2GB mobile base weights, and Gemma 3n E2B reports about a 1.91B effective-memory load. Models at the bottom of the ladder now fit in memory footprints that were unthinkable two years ago, which moves the practical question from "can I run a model locally?" to "how much of my workload have I bothered to measure against one?"
Ollama Cloud: the next rung on the same ladder
The ladder does not require a hard wall between local and cloud. The Ollama Cloud documentation states that model execution is automatically offloaded to Ollama's cloud service while callers keep using the same Ollama interfaces, with a local-only mode available to disable cloud features entirely. In the white paper's framing this is the next rung on the same ladder: cheap cloud capacity with no tooling rewrite, sitting between "larger local" and "frontier."
The pricing pages: long context is not discounted
Why does the top of the ladder cost so much? Because providers bill long context at the same per-token rate as short context. The white paper cites provider pricing pages as a timestamped configuration, fetched 2026-08-13: OpenAI's pricing and GPT-5.6 model documentation, Google's Gemini API pricing, and Anthropic's model pricing, which states the policy in one sentence: "A 900k-token request is billed at the same per-token rate as a 9k-token request." One-million-token context-window rates are published on the same page. Every cached, re-carried token of conversation history is billed at that rate every time it re-enters the window — cache-read discounts soften the unit price, but at 93.46% cache reads, volume is the dominant term.
Anthropic's own Claude Code cost documentation makes the accounting concrete with a worked session example, and notes that Claude Code computes its session cost figure locally from token counts at list price. The white paper derives from that page's worked example that 85.6% of a session's token volume is context share rather than new output — its own arithmetic on the published numbers, preserved in the research digest; the page itself does not print the percentage. The conclusion both documents point at: the cost of agentic work is dominated by context movement, so the lever is who assembles and carries the context.
How Koinessa applies it
The application is the three live mechanisms above, wired into one routing discipline.
Bounded local work first. Context compilation, drafting, log reduction, and failure classification run on the local 27B-class model at zero marginal provider cost. A fragment only escalates when the rung below it has demonstrably failed. The frontier models stay — for architectural judgment, ambiguous debugging, the long-horizon coordination AgentFloor shows they still lead on — but as leased capacity, metered and deliberate.
Local compute carries the context. Deterministic reducers assemble and compress context locally before any frontier call. That is the direct answer to the 30-day receipt: in a workload where 93.46% of 18.28 billion tokens are re-carried history, every token of context assembled or reduced locally is a token the frontier provider does not bill. The $13,910.58 figure is list-priced — an estimate at published rates for that specific window, not an invoice and not a benchmark — but the structural point survives any discount you negotiate: the volume is the problem, and the volume is context movement.
Retrieval before repetition, enforced. The standing rule is retrieval-first: the symbol index and language-server tooling for code, the shared brain's SQL and hybrid search for institutional knowledge, grep only as the declared fallback — and the rule is machine-enforced by a pre-tool hook that refuses identifier lookups by text search and answers the refusal with the exact index query to run instead. The index carries an external receipt: in a guarded sandbox probe (repository receipt, 2026-08-23, one question on one paid-tier session), the Devin CLI answered a symbol-location question through the brain's index in four seconds — 44,496 prompt tokens, 42,752 of them cache reads, 329 completion tokens, eleven steps, no text search, no source read — where the same question routed through language-server tools cost 171,317 prompt tokens over sixteen steps, roughly four times the token cost. One question, one harness, one day: a receipt, not a benchmark, and it is presented as one in the limitations and status report rather than as a promise.
This is also what keeps local-first honest as a business claim rather than a dogma. The frontier provider is still in the architecture — deliberately, at the top of a ladder, for the fraction of work that needs it. Everything below that fraction runs on customer-controlled compute the operator already owns. That is the through-line of what an AI operating system for software delivery actually is: not a replacement for frontier intelligence, but the layer that decides how little of it each task requires.
The takeaway
Cloud AI, hosted CI, and tool costs keep rising because the default architecture sends every fragment of every task to the most expensive endpoint and re-ships the entire conversation with it. The fix is not a cheaper vendor; it is a routing architecture. Decompose the workload, escalate on demonstrated capability rather than brand, let local compute carry context and verification, and reserve frontier models for the narrow band of work that measurably needs them. The receipts — a 93.46% cache-read window, a 4x token gap on one indexed lookup, a benchmark showing where small models suffice — all say the same thing: most of what you are paying frontier prices for is not frontier work. The full argument, with every qualifier and citation, is in section 6 of the Koinessa white paper.