Prompt cache warmth: show whether the next prompt is cheap

lz commented

2026-09-12 21:17:48 +02:00

Owner

A session left idle goes cold silently, and the next prompt rewrites the whole conversation prefix instead of reading it — roughly 20× the input cost, visible only as the 5h quota window moving. Nothing in Nexus distinguished the two.

What it does

Each session row gains a cache mark beside its cost and context:

$3.12 · 204k ctx · ⚡ cache 47m     warm  (--muted; amber under 10m)
$1.18 ·  88k ctx · ⚡̸ cache cold    cold  (--cold violet)
$0.07 ·   6k ctx · ○ cache —       unknown

Same three states in the terminal statusline, and on the rail hover card and session settings panel.

How

claude-code 2.1.269 already computes this — prompt_cache.expires_at arrives on the statusline payload Nexus already parses for quota. This forwards that one integer through the existing pipe; everything else is derivation.

nexus-statusline.sh → POST /api/agent/quota → quota/store → GET /api/quota → cache-warmth.ts

The measurement that shaped the design

claude-code renders the statusline on state change, not on a timer — four renders in sixty seconds of a live session. So the session that is going cold is precisely the one emitting nothing, and coldness can never be reported. Both surfaces derive it from the absolute expires_at, which is what keeps a reading correct through any amount of silence. statusLine.refreshInterval: 60 is seeded so the terminal line redraws; the browser needs nothing, it re-derives on the existing 10s poll.

Also verified against the binary rather than the docs:

The TTL is 1h and slides on every request — expires_at moved +41s across two turns. Cross-checked on six transcripts: a 3705s gap missed, 2956s hit, boundary at 3600s.
prompt_cache is an absent key until the first API response — the same unknown-vs-zero trap as rate_limits.
recache_tokens_if_cold equals context_window.total_input_tokens (delta 0, three captures), so cold prints no token count — it would restate the ctx already on the row.
last_miss_cause does not work as shipped. Rewriting a project CLAUDE.md mid-session caused no miss at all, and a forced miss (model switch) left it null. Recorded in AGENTS.md so nobody builds on it.

Cold is violet, not red

--cold: #af87ff. Cold is the resting state — every session idle an hour is cold — so red lit the entire sidebar overnight, collided with unhealthyReason in the same block, and sat beside the --running green dot on every row, the worst pairing for deuteranopia on an otherwise colour-only signal. #af87ff is xterm-256 index 141, so the terminal renders the identical colour rather than an approximation; it can't draw the zap icons, so colour is the only identity the two surfaces share.

Also in here

The statusline got 6.4× faster (276ms → 43ms per render), because refreshInterval multiplies its cost. Eight jq spawns became one @sh pass emitting named shell assignments — byte-identical output across five payloads including empty stdin and invalid JSON, and injection-safe (tested with a cwd containing a quote, a semicolon and a command).

Testing

157 files / 1689 tests, plus 42 shell cases. The derivation exists twice (TS + POSIX sh, neither can import the other), so cache-warmth-parity.test.ts runs the real script and compares its segment to cacheLabel() — the same knowing-duplication rule as mount-parity and token-scope-parity.

Every new guard was mutation-tested, and four were found incapable of failing and fixed:

guard	why it couldn't fail
floor vs round (shell)	every fixture sat on an exact minute boundary
the soon threshold	colour-only, and all assertions stripped ANSI
`-le` vs `-lt`	no fixture sat at exactly 10m
`cache_expires_at` at the endpoint	untested — wiring it to `null` left 37/37 green

The four render surfaces were verified in a real browser with computed colours, not just typechecked — this repo's vitest runs in node with no DOM, so components are structurally untestable.

Note for deploying

seedStatusLineOnBoot is one-shot and this instance's statusline_seeded flag is already set, so refreshInterval: 60 must be added by hand to statusLine in the Config Explorer. Without it the web UI is fully correct and the terminal countdown freezes. Legacy-support machinery was deliberately not built for this.

A session left idle goes cold silently, and the next prompt rewrites the whole conversation prefix instead of reading it — roughly 20× the input cost, visible only as the 5h quota window moving. Nothing in Nexus distinguished the two. ## What it does Each session row gains a cache mark beside its cost and context: ``` $3.12 · 204k ctx · ⚡ cache 47m warm (--muted; amber under 10m) $1.18 · 88k ctx · ⚡̸ cache cold cold (--cold violet) $0.07 · 6k ctx · ○ cache — unknown ``` Same three states in the terminal statusline, and on the rail hover card and session settings panel. ## How claude-code 2.1.269 already computes this — `prompt_cache.expires_at` arrives on the statusline payload Nexus **already parses for quota**. This forwards that one integer through the existing pipe; everything else is derivation. ``` nexus-statusline.sh → POST /api/agent/quota → quota/store → GET /api/quota → cache-warmth.ts ``` ## The measurement that shaped the design **claude-code renders the statusline on state change, not on a timer** — four renders in sixty seconds of a live session. So the session that is going cold is precisely the one emitting nothing, and coldness can never be *reported*. Both surfaces derive it from the absolute `expires_at`, which is what keeps a reading correct through any amount of silence. `statusLine.refreshInterval: 60` is seeded so the terminal line redraws; the browser needs nothing, it re-derives on the existing 10s poll. Also verified against the binary rather than the docs: - The TTL is **1h and slides on every request** — `expires_at` moved +41s across two turns. Cross-checked on six transcripts: a 3705s gap missed, 2956s hit, boundary at 3600s. - `prompt_cache` is an **absent key** until the first API response — the same unknown-vs-zero trap as `rate_limits`. - `recache_tokens_if_cold` **equals** `context_window.total_input_tokens` (delta 0, three captures), so cold prints no token count — it would restate the `ctx` already on the row. - **`last_miss_cause` does not work as shipped.** Rewriting a project `CLAUDE.md` mid-session caused no miss at all, and a forced miss (model switch) left it `null`. Recorded in AGENTS.md so nobody builds on it. ## Cold is violet, not red `--cold: #af87ff`. Cold is the *resting* state — every session idle an hour is cold — so red lit the entire sidebar overnight, collided with `unhealthyReason` in the same block, and sat beside the `--running` green dot on every row, the worst pairing for deuteranopia on an otherwise colour-only signal. `#af87ff` **is** xterm-256 index 141, so the terminal renders the identical colour rather than an approximation; it can't draw the zap icons, so colour is the only identity the two surfaces share. ## Also in here **The statusline got 6.4× faster** (276ms → 43ms per render), because `refreshInterval` multiplies its cost. Eight `jq` spawns became one `@sh` pass emitting named shell assignments — byte-identical output across five payloads including empty stdin and invalid JSON, and injection-safe (tested with a `cwd` containing a quote, a semicolon and a command). ## Testing 157 files / 1689 tests, plus 42 shell cases. The derivation exists twice (TS + POSIX sh, neither can import the other), so `cache-warmth-parity.test.ts` runs the real script and compares its segment to `cacheLabel()` — the same knowing-duplication rule as `mount-parity` and `token-scope-parity`. Every new guard was mutation-tested, and **four were found incapable of failing and fixed**: | guard | why it couldn't fail | |---|---| | floor vs round (shell) | every fixture sat on an exact minute boundary | | the soon threshold | colour-only, and all assertions stripped ANSI | | `-le` vs `-lt` | no fixture sat at exactly 10m | | `cache_expires_at` at the endpoint | untested — wiring it to `null` left 37/37 green | The four render surfaces were verified in a real browser with computed colours, not just typechecked — this repo's vitest runs in `node` with no DOM, so components are structurally untestable. ## Note for deploying `seedStatusLineOnBoot` is one-shot and this instance's `statusline_seeded` flag is already set, so **`refreshInterval: 60` must be added by hand** to `statusLine` in the Config Explorer. Without it the web UI is fully correct and the terminal countdown freezes. Legacy-support machinery was deliberately not built for this.

lz added 18 commits

2026-09-12 21:17:49 +02:00

docs(spec): prompt cache warmth indicator b02b153cce

claude-code 2.1.269 already emits prompt_cache on the statusline payload
Nexus consumes, so this forwards one integer (expires_at) and derives the
rest. Records what was verified against the installed binary and six
transcripts: the 1h TTL slides per request, the boundary sits at 3600s,
and the statusline renders on state change rather than on a timer --
which is why coldness is derived rather than waited for.

Folds in the jq consolidation (8 spawns to 1 via @sh named assignments,
248ms to 37ms, measured byte-identical) and a shell/TS parity test for
the duplicated derivation.

docs(plan): prompt cache warmth implementation plan a1789cd59e

Nine TDD tasks: the derivation, the field through the pipe, the client
store method, three render surfaces, the single-jq rewrite, the statusline
segment, the seeded refreshInterval, the shell/TS parity test, and the
AGENTS.md facts.

feat(quota): derive prompt cache warmth from an absolute deadline db2f639c89

feat(quota): carry prompt cache expiry from the agent callback to the API 8d011a20ee

docs(plan): correct the gate command and the type-only red state 9a22a203e3

pnpm check does not exist; the script is pnpm typecheck. And a type-only
addition cannot fail under vitest -- esbuild transpiles without
typechecking -- so the TDD red step for one has to use typecheck.

docs(plan): task 3 appends to the existing store test file 102f3bba47

session-stats.svelte.test.ts already exists (5 refcount tests) and the
convention here is .svelte.test.ts, not .test.ts -- creating a second file
would split one module's tests across two. Import style matched too: the
existing file omits the .js extension.

feat(ui): expose prompt cache warmth on the session stats store 3379d594cb

feat(ui): show prompt cache warmth on every session surface 65a13676d1

fix(ui): a cold prompt cache is violet, not danger red cd94ff4f18

Cold is the RESTING state -- every session idle an hour is cold -- so red
lit the whole sidebar overnight, which is alarm fatigue and devalues red
where it means broken (unhealthyReason renders in the same block). Red
also sat beside the --running green dot on every row, the worst pairing
for deuteranopia on an otherwise colour-only signal.

#af87ff is xterm-256 index 141, so the statusline can render the exact
same colour: the terminal cannot draw the zap icons, making colour the
only identity the two surfaces share.

perf(worker): parse the statusline payload in one jq pass c02ee5a595

feat(worker): render prompt cache warmth and report the deadline 8504dfb433

test(worker): make the cache guards able to fail e4003f4e79

Mutation testing found two of the new statusline guards could not fail.
The minute fixtures all sat on exact boundaries (2820s is 47m00s), where
flooring and rounding agree, so swapping the arithmetic to round-up left
the suite green; 47m59s separates them. And the soon threshold is
colour-only while every assertion strips ANSI, so moving it from 10 to 5
was invisible -- raw_has reads the unstripped bytes instead, which also
pins cold to 141 rather than the danger red.

Each of the four mutations now names a failing test.

feat(quota): seed a statusline refresh interval so idle sessions redraw 5360517e67

test(quota): pin the shell and TypeScript cache derivations together a45fd71887

docs(agents): record the prompt-cache payload facts 1568aea41b

What was measured against 2.1.269 rather than read from docs: the 1h TTL
slides per request with the boundary at 3600s, the statusline renders on
state change rather than on a timer (which is why coldness is derived and
never reported), recache_tokens_if_cold duplicates the context size, and
last_miss_cause does not populate even for a forced miss.

fix(test): silence no-control-regex so CI lint passes af8dfc817c

pnpm lint is the FIRST step in .forgejo/workflows/ci.yml, so this branch
would have failed the build before typecheck ever ran. Matches the
existing precedent in git-ref.ts and path-safety.ts.

The plan's verification checklist only listed typecheck and test, which
is how a required gate went unrun for nine tasks -- lint is now in it.

test(quota): cover the endpoint field and the soon-threshold boundary f8b0c7d9c7

Review found two guards that could not fail, both verified by mutation.

cache_expires_at was untested at the agent callback -- the one place in
this feature that crosses a network boundary. Wiring it to a constant
null left 37/37 green, and loosening the Zod schema to z.any() left the
endpoint's own 8 green. Every sibling field there has a negative test and
a happy-path readback; this one had neither.

And no statusline fixture sat at exactly CACHE_SOON_MINUTES, so flipping
-le to -lt kept all 41 cases green -- the same exact-boundary blind spot
that previously hid floor-vs-round.

docs: say each thing once, in AGENTS.md

ci / nexus (pull_request) Failing after 4m17s

Details

ci / images (pull_request) Has been skipped

Details

a6ef5d82dd

The 'statusline renders on state change, not a timer' argument was
re-derived in full in five places and the 'cold is not danger red' one in
three. AGENTS.md is the canonical record; the source copies now point at
it and keep only what is local and unique -- the deltaE/contrast math for
the colour choice, the rejected tab-read approach, the NaN guard.

session-stats' cache() also carried an 8-line JSDoc where its siblings
cost() and context() get two, and the reactivity note it added applies
equally to both of them.

lz added 1 commit

2026-09-15 15:12:05 +02:00

ci: install jq so the worker shell scripts under test can run

ci / nexus (pull_request) Successful in 6m15s

Details

ci / images (pull_request) Successful in 21m52s

Details

592b5428f8

nexus-statusline.sh parses its payload with jq; without jq it prints a
bare marker and exits, so cache-warmth-parity.test.ts found no cache
segment and failed all eight cases on node:22-alpine.

Adding jq also revives two suites that have never run in CI:
nexus-statusline.test.sh (42 checks) and notify-artifact.test.sh (4) both
self-skip on a missing jq and exit 0, so worker-shell-tests.test.ts
reported them green. Verified in the CI image — both pass with jq present.

ci / nexus (pull_request) Successful in 6m15s

Details

ci / images (pull_request) Successful in 21m52s

Details

This pull request can be merged automatically.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/cache-warmth:feat/cache-warmth

git switch feat/cache-warmth

Merge

Merge the changes and update on Forgejo.

git switch main

git merge --no-ff feat/cache-warmth

git switch feat/cache-warmth

git rebase main

git switch main

git merge --ff-only feat/cache-warmth

git switch feat/cache-warmth

git rebase main

git switch main

git merge --no-ff feat/cache-warmth

git switch main

git merge --squash feat/cache-warmth

git switch main

git merge --ff-only feat/cache-warmth

git switch main

git merge feat/cache-warmth

git push origin main

Rows
Columns

Prompt cache warmth: show whether the next prompt is cheap #147

What it does

How

The measurement that shaped the design

Cold is violet, not red

Also in here

Testing

Note for deploying

Checkout

Merge