Hand container health to the daemon: a Healthcheck at create, and the boot inference deleted #184

Closed
opened 2026-09-21 16:38:22 +02:00 by lz · 0 comments
Owner

Split out of #177 after a roadmap pass on 2026-09-21. It is the largest single deletion in that spec, it does not need the fleet store, and it ships while the browser still drives the poll — which leaves #177 to be about ownership rather than about health.

Why it is separable, and why it is now a deletion

docs/superpowers/specs/2026-09-17-fleet-status-layer-design.md says the healthcheck "does not delete the boot machinery". That is entirely a legacy-container argument: Healthcheck is fixed at container create, docker update cannot add one, and nothing forces a recreate — so settleBootingPhase, lastBootMarker, BOOT_GRACE_MS and the tmux execProbe all survive for workspaces created before it ships, and the spec has to carry a residual prober for the set that cannot report.

Nexus carries no backwards compatibility and updates from a clean state. So make the healthcheck universal instead of keeping two paths, and the residual class — the one the spec flags as "looks like everything working until an old workspace goes quiet" — stops being expressible rather than having to be computed.

Shape

  • Healthcheck in the create config in provisionWorker: CMD-SHELL tmux has-session -t nexus, with Interval, StartPeriod and Retries stated explicitly (see the latency trap below).
  • startWorker recreates a container whose inspected config carries no Healthcheck. The named workspace_volume survives — remove({ v: true }) reaps anonymous volumes only (fact #22) — and HostConfig.Mounts rebuilds from the frozen workers.mounts_json, which is the snapshot that exists for exactly this.
  • checkWorkerHealth reads inspect().State.Health.Status instead of exec-probing tmux. booting ends on healthy.
  • Deleted: settleBootingPhase, lastBootMarker, BOOT_GRACE_MS, the tmux probe, and the log-marker boot inference that feeds them.
  • Folded in, both named by the spec as cheaper before the store than after:
    • the dead WorkerHealthDTO fields containerState, tmuxRunning, recentLogs — zero .svelte readers (positive control: degraded hits six components). After the store they also reach GET /api/workers, which is in TOKEN_ROUTES and is projected by mcp/projection.ts, making it an API-visible removal instead of an internal one.
    • one shared execProbe, which the degraded-marker read still needs.

What fails silently if got wrong

  • Asserting the Healthcheck against an empty create config passes equally well for a form that clobbers operator mounts. Assert against a config that already holds one, following capabilities.test.ts.
  • Docker's start fires before the entrypoint's first line. /run/nexus/degraded is written at worker/entrypoint.sh:84 and tmux new-session is line 255 of 256. Nothing may be inferred from start; the degraded read stays on the poll in this slice and moves to health_status: healthy in #177.
  • An unspecified Interval defaults to 30s, so booting could clear up to 30s later than the 10s poll clears it today. State the interval and the start period, and say what the boot latency becomes.
  • A recreate that silently drops a mount. resolveSpawnMounts re-validates saved rows for a reason; the recreate path must go through the same builder rather than reconstructing HostConfig by hand.

Done when

A workspace created after this reports health from the daemon, nothing execs tmux has-session to decide workspace health, the four boot-inference helpers are gone, and no container in the fleet lacks a Healthcheck after one startWorker.

Sequencing

Independent of #178 and #179. Lands before #177, which then consumes health_status events from a fleet where every container emits them.

Split out of #177 after a roadmap pass on 2026-09-21. It is the largest single deletion in that spec, it does not need the fleet store, and it ships while the browser still drives the poll — which leaves #177 to be about *ownership* rather than about health. ## Why it is separable, and why it is now a deletion `docs/superpowers/specs/2026-09-17-fleet-status-layer-design.md` says the healthcheck "does not delete the boot machinery". That is entirely a legacy-container argument: `Healthcheck` is fixed at container create, `docker update` cannot add one, and nothing forces a recreate — so `settleBootingPhase`, `lastBootMarker`, `BOOT_GRACE_MS` and the tmux `execProbe` all survive for workspaces created before it ships, and the spec has to carry a *residual prober* for the set that cannot report. Nexus carries no backwards compatibility and updates from a clean state. So make the healthcheck universal instead of keeping two paths, and the residual class — the one the spec flags as "looks like everything working until an old workspace goes quiet" — stops being expressible rather than having to be computed. ## Shape - `Healthcheck` in the create config in `provisionWorker`: `CMD-SHELL tmux has-session -t nexus`, with `Interval`, `StartPeriod` and `Retries` stated explicitly (see the latency trap below). - `startWorker` recreates a container whose inspected config carries no `Healthcheck`. The named `workspace_volume` survives — `remove({ v: true })` reaps anonymous volumes only (fact #22) — and `HostConfig.Mounts` rebuilds from the frozen `workers.mounts_json`, which is the snapshot that exists for exactly this. - `checkWorkerHealth` reads `inspect().State.Health.Status` instead of exec-probing tmux. `booting` ends on `healthy`. - **Deleted:** `settleBootingPhase`, `lastBootMarker`, `BOOT_GRACE_MS`, the tmux probe, and the log-marker boot inference that feeds them. - Folded in, both named by the spec as cheaper before the store than after: - the dead `WorkerHealthDTO` fields `containerState`, `tmuxRunning`, `recentLogs` — zero `.svelte` readers (positive control: `degraded` hits six components). After the store they also reach `GET /api/workers`, which **is** in `TOKEN_ROUTES` and **is** projected by `mcp/projection.ts`, making it an API-visible removal instead of an internal one. - one shared `execProbe`, which the degraded-marker read still needs. ## What fails silently if got wrong - **Asserting the `Healthcheck` against an empty create config passes equally well for a form that clobbers operator mounts.** Assert against a config that already holds one, following `capabilities.test.ts`. - **Docker's `start` fires before the entrypoint's first line.** `/run/nexus/degraded` is written at `worker/entrypoint.sh:84` and `tmux new-session` is line 255 of 256. Nothing may be inferred from `start`; the degraded read stays on the poll in this slice and moves to `health_status: healthy` in #177. - **An unspecified `Interval` defaults to 30s**, so `booting` could clear up to 30s *later* than the 10s poll clears it today. State the interval and the start period, and say what the boot latency becomes. - **A recreate that silently drops a mount.** `resolveSpawnMounts` re-validates saved rows for a reason; the recreate path must go through the same builder rather than reconstructing `HostConfig` by hand. ## Done when A workspace created after this reports health from the daemon, nothing execs `tmux has-session` to decide *workspace* health, the four boot-inference helpers are gone, and no container in the fleet lacks a `Healthcheck` after one `startWorker`. ## Sequencing Independent of #178 and #179. Lands before #177, which then consumes `health_status` events from a fleet where every container emits them.
lz added this to the MCP support (#140) milestone 2026-09-21 16:38:22 +02:00
lz closed this issue 2026-09-22 11:18:51 +02:00
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
lz/agent-nexus#184
No description provided.