feat: Playwright worker image (@playwright/cli) + capability-driven variants + image/video artifacts (#2) #36

Merged
lz merged 5 commits from feat/playwright into main 2026-05-30 22:52:42 +02:00
Owner

Closes #2.

Adds an opt-in Playwright worker image so agents can navigate/screenshot apps they build, plus the generalized worker-capability rails the design called for and inline image/video artifacts. History is curated into 4 logical commits.

What's included

1. refactor(worker) — rename nexus-worker → agent-nexus-worker, node 22 base
Brand-consistent two-repo scheme (agent-nexus + agent-nexus-worker); base bumped to node:22-slim with the org.nexus.worker label. Updates WORKER_IMAGE / worker_image-setting defaults, compose, and docs.

2. feat(workers) — label-driven capability registry + per-workspace image selection
Worker images self-describe via OCI labels; a pure capability registry (nexus/src/lib/server/workers/capabilities.ts) resolves per-image HostConfig deltas (Init + ShmSize) and claude launch-flag deltas — the registry, not the path-only socket proxy, is the security boundary. Adds the workers.worker_image column (migration 0010), a label-filtered GET /api/worker-images, spawn-time validation, an image picker in the create-workspace form, and the imageLaunchArgs plumbing.

3. feat(artifacts) — inline image/video artifacts
Widens artifacts.kind to image|video (migration 0011 rebuilds the table — SQLite can't ALTER a CHECK), classifies by extension (inferKind + MIME helper), adds a raw media streaming endpoint (honors max_file_bytes, 413 on oversize), and renders <img>/<video> inline in the Artifacts panel.

4. feat(worker) — Playwright image via @playwright/cli + per-image plugin skill
Headless Chromium baked via @playwright/cli's bundled core. No MCP server, port, or claude mcp add-json registration — the agent drives the browser per-command (playwright-cli open|goto|snapshot|click|screenshot|…). @playwright/cli's SKILL is baked as a claude plugin at /opt/nexus/plugins/playwright and loaded per-session via claude --plugin-dir (contributed by the capability registry's launchArgs). It lives outside ~/.claude, so it reaches only this image's sessions and never the fleet-shared config volume. CI builds the variant (tag suffix -playwright0.1.13).

Note: @playwright/cli vs @playwright/mcp

This PR originally used @playwright/mcp (an always-on HTTP MCP server). It now uses Microsoft's @playwright/cli — their recommended browser-automation surface for coding agents: cleaner (no server/port/registration), more token-efficient, ships a maintained skill, and sidesteps the HTTP-transport MCP stability risk. Both bundle the same alpha playwright-core, so the browser-version story is unchanged: one baked alpha Chromium for the agent; a project's stable npx playwright test does a one-time npx playwright install chromium.

Verification

  • ✅ typecheck (0 errors / 895 files), lint clean, 524 tests pass, build OK
  • ✅ adversarial multi-lens review (integration / docker-build / security / completeness), every finding re-verified → zero confirmed issues
  • ✅ curated history's final tree is byte-identical to the fully-gated pre-curation commit

Remaining manual smoke (Docker host / CI)

  • Build both images (CI images matrix exercises the buildx base-pre-build for the Playwright FROM).
  • Spawn a Playwright workspace → confirm playwright-cli --help works, the playwright-cli skill is discoverable (loaded via --plugin-dir, not in ~/.claude), a screenshot round-trips through notify-artifact, and /dev/shm sizing (1 GiB) holds on a non-trivial page.
Closes #2. Adds an **opt-in Playwright worker image** so agents can navigate/screenshot apps they build, plus the generalized worker-capability rails the design called for and inline image/video artifacts. History is curated into 4 logical commits. ## What's included **1. `refactor(worker)` — rename `nexus-worker` → `agent-nexus-worker`, node 22 base** Brand-consistent two-repo scheme (`agent-nexus` + `agent-nexus-worker`); base bumped to `node:22-slim` with the `org.nexus.worker` label. Updates `WORKER_IMAGE` / `worker_image`-setting defaults, compose, and docs. **2. `feat(workers)` — label-driven capability registry + per-workspace image selection** Worker images self-describe via OCI labels; a pure capability registry (`nexus/src/lib/server/workers/capabilities.ts`) resolves per-image **HostConfig deltas** (`Init` + `ShmSize`) *and* **claude launch-flag deltas** — the registry, **not** the path-only socket proxy, is the security boundary. Adds the `workers.worker_image` column (migration 0010), a label-filtered `GET /api/worker-images`, spawn-time validation, an image picker in the create-workspace form, and the `imageLaunchArgs` plumbing. **3. `feat(artifacts)` — inline image/video artifacts** Widens `artifacts.kind` to `image|video` (migration 0011 rebuilds the table — SQLite can't `ALTER` a `CHECK`), classifies by extension (`inferKind` + MIME helper), adds a raw media streaming endpoint (honors `max_file_bytes`, 413 on oversize), and renders `<img>`/`<video>` inline in the Artifacts panel. **4. `feat(worker)` — Playwright image via `@playwright/cli` + per-image plugin skill** Headless Chromium baked via `@playwright/cli`'s bundled core. **No MCP server, port, or `claude mcp add-json` registration** — the agent drives the browser per-command (`playwright-cli open|goto|snapshot|click|screenshot|…`). `@playwright/cli`'s SKILL is baked as a **claude plugin** at `/opt/nexus/plugins/playwright` and loaded per-session via `claude --plugin-dir` (contributed by the capability registry's `launchArgs`). It lives **outside `~/.claude`**, so it reaches only this image's sessions and never the fleet-shared config volume. CI builds the variant (tag suffix `-playwright0.1.13`). ## Note: `@playwright/cli` vs `@playwright/mcp` This PR originally used `@playwright/mcp` (an always-on HTTP MCP server). It now uses Microsoft's `@playwright/cli` — their recommended browser-automation surface for coding agents: cleaner (no server/port/registration), more token-efficient, ships a maintained skill, and sidesteps the HTTP-transport MCP stability risk. Both bundle the **same alpha `playwright-core`**, so the browser-version story is unchanged: one baked alpha Chromium for the agent; a project's stable `npx playwright test` does a one-time `npx playwright install chromium`. ## Verification - ✅ typecheck (0 errors / 895 files), lint clean, **524 tests pass**, build OK - ✅ adversarial multi-lens review (integration / docker-build / security / completeness), every finding re-verified → **zero confirmed issues** - ✅ curated history's final tree is byte-identical to the fully-gated pre-curation commit ## Remaining manual smoke (Docker host / CI) - Build both images (CI `images` matrix exercises the buildx base-pre-build for the Playwright `FROM`). - Spawn a Playwright workspace → confirm `playwright-cli --help` works, the `playwright-cli` skill is discoverable (loaded via `--plugin-dir`, not in `~/.claude`), a screenshot round-trips through `notify-artifact`, and `/dev/shm` sizing (1 GiB) holds on a non-trivial page.
lz force-pushed feat/playwright from fdc74d4f3d
Some checks failed
ci / nexus (pull_request) Successful in 2m15s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 3m47s
ci / images (./worker, agent-nexus-worker) (pull_request) Successful in 3m13s
ci / images (MCP_VERSION=0.0.75, ./worker, Dockerfile.playwright, agent-nexus-worker, -playwright0.0.75) (pull_request) Failing after 2m31s
to 2c9d2ebec2
Some checks failed
ci / nexus (pull_request) Successful in 2m49s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 3m59s
ci / images (./worker, agent-nexus-worker) (pull_request) Successful in 1m16s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, Dockerfile.playwright, agent-nexus-worker, -playwright0.1.13) (pull_request) Failing after 2m15s
2026-05-30 17:39:53 +02:00
Compare
lz changed title from feat: Playwright worker image + capability-driven variants + image/video artifacts (#2) to feat: Playwright worker image (@playwright/cli) + capability-driven variants + image/video artifacts (#2) 2026-05-30 17:40:19 +02:00
lz force-pushed feat/playwright from 2c9d2ebec2
Some checks failed
ci / nexus (pull_request) Successful in 2m49s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 3m59s
ci / images (./worker, agent-nexus-worker) (pull_request) Successful in 1m16s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, Dockerfile.playwright, agent-nexus-worker, -playwright0.1.13) (pull_request) Failing after 2m15s
to 05dc4d3cd3
Some checks failed
ci / nexus (pull_request) Successful in 2m37s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 52s
ci / images (./worker, agent-nexus-worker) (pull_request) Successful in 48s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, Dockerfile.playwright, agent-nexus-worker, -playwright0.1.13) (pull_request) Failing after 3m59s
2026-05-30 17:57:47 +02:00
Compare
lz force-pushed feat/playwright from 05dc4d3cd3
Some checks failed
ci / nexus (pull_request) Successful in 2m37s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 52s
ci / images (./worker, agent-nexus-worker) (pull_request) Successful in 48s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, Dockerfile.playwright, agent-nexus-worker, -playwright0.1.13) (pull_request) Failing after 3m59s
to 3dcfc98375
Some checks failed
ci / nexus (pull_request) Successful in 2m23s
ci / images (pull_request) Failing after 1m23s
2026-05-30 18:17:29 +02:00
Compare
lz force-pushed feat/playwright from 3dcfc98375
Some checks failed
ci / nexus (pull_request) Successful in 2m23s
ci / images (pull_request) Failing after 1m23s
to cd50ddb08e
Some checks failed
ci / nexus (pull_request) Successful in 2m46s
ci / images (pull_request) Failing after 1m5s
2026-05-30 18:33:50 +02:00
Compare
lz force-pushed feat/playwright from cd50ddb08e
Some checks failed
ci / nexus (pull_request) Successful in 2m46s
ci / images (pull_request) Failing after 1m5s
to 9f2085961a
All checks were successful
ci / nexus (pull_request) Successful in 2m39s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 4m20s
ci / images (./worker, agent-nexus-worker, base) (pull_request) Successful in 1m9s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, agent-nexus-worker, -playwright0.1.13, playwright) (pull_request) Successful in 6m33s
2026-05-30 18:48:49 +02:00
Compare
lz force-pushed feat/playwright from 9f2085961a
All checks were successful
ci / nexus (pull_request) Successful in 2m39s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 4m20s
ci / images (./worker, agent-nexus-worker, base) (pull_request) Successful in 1m9s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, agent-nexus-worker, -playwright0.1.13, playwright) (pull_request) Successful in 6m33s
to f97dc34d03
All checks were successful
ci / nexus (pull_request) Successful in 3m11s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 1m22s
ci / images (./worker, agent-nexus-worker, base) (pull_request) Successful in 1m11s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, agent-nexus-worker, -playwright0.1.13, playwright) (pull_request) Successful in 7m0s
2026-05-30 21:10:55 +02:00
Compare
feat(media): configurable media cap + shared image/video viewer
All checks were successful
ci / nexus (pull_request) Successful in 3m9s
ci / images (./nexus, agent-nexus) (pull_request) Successful in 4m20s
ci / images (./worker, agent-nexus-worker, base) (pull_request) Successful in 1m18s
ci / images (PLAYWRIGHT_CLI_VERSION=0.1.13, ./worker, agent-nexus-worker, -playwright0.1.13, playwright) (pull_request) Successful in 1m13s
96d47d180b
Two issues surfaced by Playwright smoke-testing:

1. notify-artifact rejected normal screenshots — the 256 KiB max_file_bytes
   text cap was wrongly applied to binary media. Add a dedicated, operator-
   configurable media cap: `media_max_file_bytes` instance setting (env
   MEDIA_MAX_FILE_BYTES, default 10 MiB, range 1 KiB–100 MiB). The pure
   maxBytesForKind() picks the cap by kind; applied at artifact registration,
   the artifact raw-serve endpoint, and the worker file-explorer read.

2. The file explorer couldn't view image/video. Render them inline, reusing
   the Artifacts pane's renderer:
   - NEW $lib/media.ts — one client-safe ext->{kind,mime} classifier; the
     server mime.ts (mimeFromPath) and service.ts (inferKind) import it too, so
     the image/video lists can't drift between client and server.
   - NEW MediaView.svelte — shared <img>/<video> renderer used by BOTH
     ArtifactViewer (src = raw endpoint) and FileExplorerPanel (src = data: URL
     built from the file bytes, read with allowBinary).

Settings UI gets a friendly label + help for the new cap. config-explorer
reads are uncapped (only writes limit size) so its preview works unchanged.

typecheck 0, lint clean, 532 tests, build OK; adversarial review clean
(only cosmetic nits, addressed; pre-existing SVG-in-<img> note left as-is).
lz merged commit 46b6cd0ed2 into main 2026-05-30 22:52:42 +02:00
lz deleted branch feat/playwright 2026-05-30 22:52:43 +02:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
lz/agent-nexus!36
No description provided.