Optional: announce a SIGKILLed claude from the launch line (the #177c wrapper, descoped) #194

Open
opened 2026-09-22 13:39:57 +02:00 by lz · 0 comments
Owner

Split out of #177. #177c shipped the floor — server-side claude liveness, observed in the exec pass convergeSessions already makes (probeLiveClaudeCwds -> sessions.claude_alive). This issue is the latency optimisation that was originally specified in its place, kept because it is still defensible and because the measurements below should not have to be redone.

What is already covered, and by what

how claude dies reported today by
/exit, logout yes, immediately SessionEnd hook
SIGTERM (143) yes, immediately SessionEnd, reason other
SIGHUP (129) yes, immediately SessionEnd, reason other
uncaught exception (1) yes, immediately SessionEnd, reason other
SIGKILL / SIGSEGV / OOM only on the next converge pass claude_alive observation
hung, or lost its pty only on the next converge pass claude_alive observation

So the gap a launch-line reporter would close is latency on the bottom two rows, bounded today by RESYNC_INTERVAL_MS (60s).

Measured, against the installed claude 2.1.278 — do not re-derive

  • SessionEnd is not graceful-exit-only. shutdown(e = 0, n = "other", r) awaits executeSessionEndHooks before forceExit; process.on("SIGTERM") -> shutdown(143, "other"), SIGHUP -> shutdown(129), uncaughtException -> shutdown(1). Driven end to end: a real claude --print SIGTERM'd at 6s exits 143 and writes {"hook_event_name":"SessionEnd","reason":"other"}. other is already in the plugin's matcher. Negative control: SIGKILL at 1.2s fires nothing. AGENTS.md fact #26 has been corrected.
  • The full hook event set in that build contains no crash or abnormal-exit event. 33 events; none of them fire for SIGKILL.
  • The wrapper works, and $? carries the code. /bin/sh -c "cd $W && claude ...; echo WRAPPER_SAW=$? >> log" gave WRAPPER_SAW=0 on a clean run and WRAPPER_SAW=137 under kill -9, with no SessionEnd file in the second case. spawnSessionPane already passes ['/bin/sh', '-c', launchCmd] as argv, and "$?" inside a JS template literal is literal.
  • It cannot see seven cases, four of which are Nexus acting deliberately: restartSession (stop and restart, both respawn-window -k), closeWorkspaceSession, openWorkspaceSession onto an existing window, deleteSession (kill-window), container stop/kill (PID 1 is exec tmux attach, so the daemon SIGKILLs the rest), tmux server death, and Nexus being unreachable.
  • An old worker image degrades cosmetically, not fatally. A missing /usr/local/bin/nexus-report-exit gives one not found line on the pane's stderr; the wrapper continues to exec bash and exits 0. Visible on every session of every un-recreated workspace until the image release lands.
  • tmux's pane-died hook is not a substitute. With remain-on-exit on it does deliver the code (pane_dead_status=137) and the window stays in list-windows, so convergence is undisturbed — but run-shell does not inherit the window's -e environment, so NEXUS_SESSION_ID is unset and the hook cannot say which session died. spawnSessionPane sets that per window precisely for attribution. It would also cost dropping ; exec bash.

Three things to get right

  1. Capture rc=$? BEFORE any guard command. Reversed, command -v resets $? and every crash reports 0 — silently.
  2. The rule is "exited, and no SessionEnd accounted for it", never "exit code non-zero". 143 and 129 are orderly shutdowns that already wrote attention_state='ended'; overwriting that with a fault paints a deliberate stop as a crash, which is the distinction fact #26 exists to keep.
  3. The route must join AGENT_CALLBACKS (fact #23) or every report 401s at the gate while the producer swallows the status — how the attention feature shipped dead once already. hooks.server.test.ts walks src/routes/api/agent/ in both directions, so adding the route tells you if you forgot.

Cost

A worker image release, which is owed anyway (worker/entrypoint.sh changed in #184). Check whether #183 is close enough to release once for all of them.

Not worth doing if

The 60s resync is fast enough for a SIGKILLed agent. It probably is for a single operator — which is why this is filed rather than built.

Split out of #177. **#177c shipped the floor** — server-side claude liveness, observed in the exec pass `convergeSessions` already makes (`probeLiveClaudeCwds` -> `sessions.claude_alive`). This issue is the *latency* optimisation that was originally specified in its place, kept because it is still defensible and because the measurements below should not have to be redone. ## What is already covered, and by what | how claude dies | reported today | by | |---|---|---| | `/exit`, logout | yes, immediately | `SessionEnd` hook | | SIGTERM (143) | yes, immediately | `SessionEnd`, reason `other` | | SIGHUP (129) | yes, immediately | `SessionEnd`, reason `other` | | uncaught exception (1) | yes, immediately | `SessionEnd`, reason `other` | | **SIGKILL / SIGSEGV / OOM** | **only on the next converge pass** | `claude_alive` observation | | hung, or lost its pty | only on the next converge pass | `claude_alive` observation | So the gap a launch-line reporter would close is **latency on the bottom two rows**, bounded today by `RESYNC_INTERVAL_MS` (60s). ## Measured, against the installed claude 2.1.278 — do not re-derive - **`SessionEnd` is not graceful-exit-only.** `shutdown(e = 0, n = "other", r)` `await`s `executeSessionEndHooks` before `forceExit`; `process.on("SIGTERM")` -> `shutdown(143, "other")`, SIGHUP -> `shutdown(129)`, `uncaughtException` -> `shutdown(1)`. Driven end to end: a real `claude --print` SIGTERM'd at 6s exits 143 **and** writes `{"hook_event_name":"SessionEnd","reason":"other"}`. `other` is already in the plugin's matcher. Negative control: SIGKILL at 1.2s fires nothing. AGENTS.md fact #26 has been corrected. - **The full hook event set in that build contains no crash or abnormal-exit event.** 33 events; none of them fire for SIGKILL. - **The wrapper works, and `$?` carries the code.** `/bin/sh -c "cd $W && claude ...; echo WRAPPER_SAW=$? >> log"` gave `WRAPPER_SAW=0` on a clean run and `WRAPPER_SAW=137` under `kill -9`, with no SessionEnd file in the second case. `spawnSessionPane` already passes `['/bin/sh', '-c', launchCmd]` as argv, and `"$?"` inside a JS template literal is literal. - **It cannot see seven cases**, four of which are Nexus acting deliberately: `restartSession` (stop and restart, both `respawn-window -k`), `closeWorkspaceSession`, `openWorkspaceSession` onto an existing window, `deleteSession` (`kill-window`), container stop/kill (PID 1 is `exec tmux attach`, so the daemon SIGKILLs the rest), tmux server death, and Nexus being unreachable. - **An old worker image degrades cosmetically, not fatally.** A missing `/usr/local/bin/nexus-report-exit` gives one `not found` line on the pane's stderr; the wrapper continues to `exec bash` and exits 0. Visible on every session of every un-recreated workspace until the image release lands. - **tmux's `pane-died` hook is not a substitute.** With `remain-on-exit on` it does deliver the code (`pane_dead_status=137`) and the window stays in `list-windows`, so convergence is undisturbed — but `run-shell` does **not** inherit the window's `-e` environment, so `NEXUS_SESSION_ID` is unset and the hook cannot say which session died. `spawnSessionPane` sets that per window precisely for attribution. It would also cost dropping `; exec bash`. ## Three things to get right 1. **Capture `rc=$?` BEFORE any guard command.** Reversed, `command -v` resets `$?` and every crash reports 0 — silently. 2. **The rule is "exited, and no `SessionEnd` accounted for it", never "exit code non-zero".** 143 and 129 are orderly shutdowns that already wrote `attention_state='ended'`; overwriting that with a fault paints a deliberate stop as a crash, which is the distinction fact #26 exists to keep. 3. **The route must join `AGENT_CALLBACKS`** (fact #23) or every report 401s at the gate while the producer swallows the status — how the attention feature shipped dead once already. `hooks.server.test.ts` walks `src/routes/api/agent/` in both directions, so adding the route tells you if you forgot. ## Cost A worker image release, which is owed anyway (`worker/entrypoint.sh` changed in #184). Check whether #183 is close enough to release once for all of them. ## Not worth doing if The 60s resync is fast enough for a SIGKILLed agent. It probably is for a single operator — which is why this is filed rather than built.
lz added this to the MCP support (#140) milestone 2026-09-22 13:39:57 +02:00
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
lz/agent-nexus#194
No description provided.