The 2.4-billion-token retro · Kasra Aminiyeganeh

Ten minutes after I asked a room full of agents to find out where my tool gets in the way, my tool got in their way.

The session running the audit had just launched its workflow and tried to end its turn. The stop hook said no:

SAKO gate: 1 problem(s) before you stop:
  Ledger: T-64 is claimed by sk-1005-3b773bbf7caf, which is not a live session; resume it, or take it over: claim T-64 --takeover REASON

(A stop hook runs whenever an agent tries to finish. Mine blocks once and says what’s missing, then lets the next attempt through, so it can never trap an agent.) T-64 was a task another session had claimed and abandoned almost two weeks earlier. The session trying to stop had never touched it. I typed “take over T-64, the old session is gone” and moved on.

Hours later, the agents’ report put that exact move near the top. Sessions kept getting blocked over someone else’s stale claim and grabbing work they didn’t want just to be allowed to stop. I’d done it myself before the report was even written.

The whole run, from that audit to a new release, went through about 2.4 billion tokens, or about $859 at API prices. Nearly all of it was agents rereading their own context.

Ten days of logs

SAKO is my small task ledger and finish gate for Claude Code and Codex, one Python file plus a few hooks. Sessions claim tasks in a shared Markdown file. When one tries to stop, the hook checks its changes against its claims, and closing a task writes a receipt saying what was checked at which commit. My last post ended by asking where it gets in your way. Ten days of real work later, the answer was sitting in my own logs.

The work was a demo of Crexil, my app that sends one brief to several AI models, and it was messier than a coding sprint. Coordinator sessions handed fixes to about 45 implementer agents in their own worktrees. Critic agents picked apart a video cut, and other sessions sat with me for hours of back-and-forth. All of it ran on SAKO, and all of it left transcripts, 1.9 GB of them.

So I asked for a retro. Where was SAKO overhead, and where did a hook fail or not fit the work? Every claim had to point at the transcript line it came from.

Reading the logs twice

No agent can hold 1.9 GB, so the session first boiled each transcript down to a timeline. Then it wrote the retro as a workflow, a Claude Code feature where a short script launches agents in stages and hands each stage’s results to the next:

transcripts
  -> digest script   one timeline per session
  -> miners          observations tied to transcript lines
  -> 9 lenses        hooks, overhead, coordination...
  -> merge, gaps     duplicates out, holes filled
  -> skeptics        two per finding: one rereads the
                     evidence, one checks the fix
  -> report          94 findings, a keep list, a backlog

It took 245 agents and three and a half hours. Every finding survived in some form, but the skeptics struck about 40 claims and proposed fixes inside them and corrected the counts on 86 of the 94. Some corrections were small. Forked sessions copy their parent’s records, so the first count found 685 stops where there were 634. Some changed the story. One finding said a quarter of every session’s start-up context was noise, and the skeptic got about a tenth.

They caught me too. A week earlier, my own note on the worst afternoon said the gate had failed 15 of 15 times. The skeptics recounted. It was 15 of 35. My script had skipped the rows where passing stops get recorded, so it only ever saw failures.

None of this is new. It’s a postmortem where someone checks your arithmetic, except I could afford that someone on every finding instead of just the ones I already doubted. (They run on the same model as the miners, so they share its blind spots. Great at recounting, weaker at noticing what nobody looked at.)

The gate that looked alive

The headline finding was ugly. Counted once each, 122 of 574 stops never ran the gate at all, about one in five. That afternoon was just the worst of it.

In my setup, the .sako records folder is its own Git repository with a private remote. When a session’s shell ended a turn inside it (say, right after cd .sako to commit records), the hook asked Git for the project, got the inner repo and found no kit. No kit means the hook lets the stop through, per the never-trap rule. The agent never knew. The only error was in a terminal I wasn’t watching. And since it failed only some of the time, status kept showing a fresh “last ran” time. It looked alive.

I’d noticed the symptom mid-project and told the agents, in a memory note, never to cd into .sako. The skips stopped. The retro found where that sentence went next. Agents had copied it into 28 subagent briefs, a skill, an agent definition and a script, and in the subagent briefs it protected nothing, because subagents don’t run the stop hook.

SAKO exists because I stopped trusting rules held up by nothing but a sentence. This one I wrote myself, and agents dutifully copied it everywhere.

The real fix looks in the current folder first and falls back to the project folder Claude Code reports. The two obvious fixes, tried first in throwaway repos, both broke something. Finding the kit through the project folder alone blocked sessions that had done nothing wrong, and moving the whole gate there let a worktree change slip through. A test covers the nested case now, and the docs admit that a fresh “last ran” doesn’t prove every stop was checked.

The opening’s stale-claim block got fixed too. A session is now blocked only by findings about its own work, and someone else’s stale claim is just a note:

SAKO gate: clear in /tmp/tmp.2zGyrH9kn3/demo
note: T-1 is claimed by sk-1019-2bd806c97f0e, which is not a live session; resume it, or take it over (claim T-1 --takeover REASON), then finish or park it (edit T-1 --park REASON)

The retro also listed what worked, so no later trim could touch it. The hook typically ran in about 85 milliseconds, and refusals named their fix. The claim check stopped two real collisions, and both times the second agent messaged the holder instead of barging in.

Agents rebuilding their own gate

I read the report past midnight and told the session to “go with your recoms and best judgement” and fix it all as the coordinator.

It turned the findings into tasks and nine groups of fixes, each built by an agent in its own worktree and attacked by a reviewer before landing. Every decision it made in my name went into a log for me to review. And SAKO gated all of it. The session claimed its tasks in SAKO’s own ledger and got checked at every stop like anyone else, closing 25 tasks with receipts.

SAKO’s runtime was capped at 1,800 lines back then, to keep it readable, and the fixes pushed it to 1,850. Trimming 50 lines of freshly reviewed code is an easy way to break things, so an agent recorded every command-line call the test suite makes, replayed them against both versions and compared the results. Same output. That covers what the tests exercise, not every path, but it beats eyeballing a diff.

It never runs out of findings

Then came the reviews, and they kept finding real problems.

One noticed the tests would fail on GitHub’s servers. A new test dropped a timestamp into a regex without escaping it. My machine writes the zone as -0700, which is harmless. CI runs in UTC, so it writes +0000, and + means “one or more”. Green on my laptop, red on every runner. The fix was one call:

- rf"... since {since}; closed {MINUTE}; ..."
+ rf"... since {re.escape(since)}; closed {MINUTE}; ..."

The release review found a real hole. An agent could edit a file inside a task another session had just closed, and the edit slipped through unrecorded. The repair added a rule that a closed task stops covering a file changed after the minute it closed. The recheck found the rule ignored time zones, so a close written in Pacific time read as a different minute on a UTC machine. Another repair, another agent probing for holes.

Every round found something real, and every round was partly about the last round’s repair. Each repair was new code for the next reviewer, so in this run the loop showed no sign of settling on its own.

More than a day in, I wrote “make it happen faster, no more final final rework”. The probe came back anyway, with one more. If a formatter, a stash or an editor rewrites a closed file later, the new rule flags it as unrecorded work, and in a shared checkout it can stop a session that never touched the file. Yes, the same kind of block this post opened with.

Ship or fix? My rule is short.

Ship what’s left when it fails safe. Fix it first when it fails open.

This one fails safe, since it never lets unrecorded work through. The owner clears it with a commit it should make anyway, and a bystander’s second stop goes through like every second stop. False blocks aren’t free (the opening shows what they teach agents), but this one is narrow. It shipped under “Known issues”. Later that day, 0.7.0 kept the block and fixed its advice, so the finding now names the task that closed the file.

I steered the whole run in about a dozen messages, four of them asking where things stood and one of them “ok”.

2.4 billion tokens, itemized

Here’s the bill, counted from the usage block the API returns on every request, for the main session and all 335 agents (245 in the retro, the rest in the rebuild and reviews):

	Tokens	At API rates
Cache reads	2,303,009,043	$461
Cache writes	62,400,086	$316
Output, thinking included	4,108,989	$82
Uncached input	31,382	$0.13
Total	2,369,549,500	$859

Those are Claude Opus 5.5’s published API prices per million tokens as of October 2026. Input is $4, output $20 and cache reads $0.20, and cache writes are $5 or $8 for the five-minute or one-hour kind (60.9 million and 1.5 million here). My subscription covered the run, so there was no $859 invoice. It’s what the same usage would cost on the API.

Cache reads were 97% of the tokens. That’s agents rereading their own context, about 150,000 tokens of it on every request. At a twentieth of the input price, caching kept that cheap. Without it, the same run would have cost about $9,500. Output, the part that actually writes code and findings, was about a tenth of the money.

The retro was a bit over half the bill, and a review round ran $25 to $120. Now I know what “one more pass” costs.

To count your own sessions, mind one trap. Claude Code logs each request on several lines, and the early ones carry a partial output count. Sum every line and you roughly double your total. Take the first line per request and you miss most of the output. Keep the block with the largest output count per request ID, across the session file and its subagents/ folder:

import json

usage = {}  # one dict for the session file and every file under its subagents/ folder
for path in paths:
    for line in open(path):
        rec = json.loads(line)
        u = rec.get("message", {}).get("usage") if rec.get("type") == "assistant" else None
        key = rec.get("requestId")
        if u and key and u["output_tokens"] >= usage.get(key, {}).get("output_tokens", -1):
            usage[key] = u

Then sum the four token fields. On this session that gives exactly 2,369,549,500.

Where it stops

This is one person’s project, audited by the same kind of model that helped build it. The counts come from private transcripts full of unrelated work, so what you can check is the code. The fixes and their tests are in the 0.6.0 release, with a changelog that owns up to the known issue. I didn’t run a smaller retro to compare, and I haven’t measured the skip rate since the release.

The evidence is perishable, too. My last message in the session was “Can you shut down the pc?”, and the reboot wiped the scratch folder holding the retro’s scripts. I had to dig them back out of the transcripts, which Claude Code deletes after 30 days by default.

And my part isn’t done. I gave the go to publish before the last workflow ended, without reading every line of the diff. The tests and the reviewer agents went through it. The log of decisions made in my name is next on my list. Calling the stop and reading that log are the two jobs I couldn’t hand off.

SAKO is on PyPI, now at 0.7.1. uvx [email protected] init installs it, and the changelog has the upgrade steps and everything that changed since this run.