Add operator-approved port forwarding so dev servers running in worker containers can be opened from the host #3

Closed
opened 2026-05-12 18:32:11 +02:00 by lz · 1 comment
Owner

Motivation

When an agent runs npm run dev inside a worker container, the operator currently has no way to open the dev server in a browser. This issue adds:

  1. A worker-reachable Nexus API so agents can self-register "I have a dev server on port X" — already-on-bridge containers POST to Nexus via Docker DNS, no host port published.
  2. A host-side TCP listener that pipes traffic to the container's bridge IP on the agent-declared port. Started only after the operator clicks Approve.
  3. Operator-facing UI to list pending/approved ports, approve/revoke them, and copy the host URL to open in a new browser tab.

Companion to #2 (Playwright MCP for agents). This issue is purely the operator-side preview channel; the agent's own browser access is independent.

Threat model

  • Single-operator design carries over. Host listener binds 127.0.0.1 only — anyone with loopback on the Nexus host already has equivalent access to the UI.
  • Two-step gate: agent declares port → row is pending → operator approves → listener opens. No auto-forwarding ever.
  • Agent callback endpoints (/api/agent/*) are internal-only. They are reachable from the shared Docker bridge but are never published on a host port. Primary defense: don't expose them outside the bridge. Defense in depth: a Fastify route-prefix that returns 403 unless req.ip is in a Docker bridge subnet.
  • Idle timeouts on each proxied connection (5 min default) prevent connection leaks. The TCP proxy doesn't parse HTTP so Slowloris isn't a relevant attack surface.
  • Listeners torn down on container die. Hook into the existing health-poll rather than subscribing to Docker events (avoids needing EVENTS=1 on docker-socket-proxy).

Verified facts (corrections to the original proposal)

  • NetworkSettings.IPAddress (top-level) is only populated on the legacy default bridge. Compose-managed networks need NetworkSettings.Networks[<networkName>].IPAddress. Use Object.values(info.NetworkSettings.Networks)[0].IPAddress. Don't cache the result — bridge IPs reassign on restart.
  • The existing docker-socket-proxy config in docker-compose.yml#L10 already sets CONTAINERS=1, which does cover GET /containers/{id}/json. No new socket-proxy permissions are needed. NETWORKS=1 is only for the separate /networks/* endpoints, which we don't use.
  • Raw net.createServer + socket.pipe() is fully transparent at L4 — WebSocket upgrades, HMR, and long-lived connections all work. Don't reach for http-proxy / http-proxy-middleware; they parse HTTP and break Next.js HMR.
  • dockerode 5.0.0 is current and docker.getContainer(id).inspect() returns the full NetworkSettings we need.
  • Vite caveat: Vite ≥ 6 will reject requests with an unexpected Host header (Blocked request. This host is not allowed.) unless server.allowedHosts lists it. Operator needs to set server.allowedHosts: ['127.0.0.1'] in their vite.config.js. Next.js dev has no such issue.

Suggested implementation

1. Worker-reachable Nexus API

Nexus and worker containers both attach to the default bridge (docker-compose.yml#L46-L48), so workers can already reach agent-nexus:3001 via Docker DNS. Two things to add:

  • Bind Fastify on 0.0.0.0:3001 (or add a second listener bound to the bridge IP). The host publish in compose stays 127.0.0.1:3001:3001don't publish a second host port. This keeps the agent endpoints reachable from inside the bridge but not from outside.
  • Agent-callback route prefix /api/agent/*, no requireUnlocked guard. Add a small preHandler that returns 403 unless req.ip is in a private/bridge range (e.g. 172.16.0.0/12, 192.168.0.0/16). Defense in depth — primary defense is the network policy above.

A small shell helper baked into the worker image (worker/notify-preview.sh/usr/local/bin/notify-preview):

#!/bin/bash
# Usage: notify-preview <port> [label]
PORT=$1
LABEL=${2:-"dev server"}
curl -sf -X POST "http://agent-nexus:3001/api/agent/previews" \
  -H "Content-Type: application/json" \
  -d "{\"worker_id\":\"$WORKER_ID\",\"container_port\":$PORT,\"label\":\"$LABEL\"}" \
  && echo "Preview registered: port $PORT" \
  || echo "Failed to notify Nexus (non-fatal)"

The worker spawn in nexus/src/workers/service.ts#L139 needs to add WORKER_ID=<id> to the container's Env: so the helper has it. Not a secret — labels and inspect already expose it.

2. Schema

New migration nexus/migrations/000N_preview_ports.ts:

await db.schema
  .createTable('preview_ports')
  .addColumn('id', 'text', c => c.primaryKey())
  .addColumn('worker_id', 'text', c =>
    c.notNull().references('workers.id').onDelete('cascade'))
  .addColumn('container_port', 'integer', c => c.notNull())
  .addColumn('host_port', 'integer')
  .addColumn('status', 'text', c => c.notNull()) // 'pending' | 'approved' | 'revoked'
  .addColumn('label', 'text')
  .addColumn('created_at', 'integer', c => c.notNull())
  .addColumn('approved_at', 'integer')
  .execute()

await db.schema
  .createIndex('preview_ports_worker_port_uq')
  .on('preview_ports').columns(['worker_id', 'container_port']).unique()
  .execute()

await db.schema
  .createIndex('preview_ports_host_port_uq')
  .on('preview_ports').column('host_port').unique()
  .where('host_port', 'is not', null)
  .execute()

3. TCP proxy service

New nexus/src/preview/tcp-proxy.ts:

import net from 'node:net'
import type { Container } from 'dockerode'

const OPEN: Map<string, net.Server> = new Map()  // key = `${workerId}:${port}`
const IDLE_MS = 5 * 60 * 1000

async function resolveContainerIp(container: Container): Promise<string> {
  const info = await container.inspect()
  const network = Object.values(info.NetworkSettings.Networks)[0]
  if (!network?.IPAddress) throw new Error('no bridge IP')
  return network.IPAddress
}

export async function start(workerId: string, containerId: string,
                            containerPort: number, hostPort: number) {
  const server = net.createServer({ allowHalfOpen: false }, async (client) => {
    try {
      const ip = await resolveContainerIp(docker.getContainer(containerId))
      const upstream = net.connect(containerPort, ip)
      const teardown = () => { client.destroy(); upstream.destroy() }
      client.on('error', teardown); upstream.on('error', teardown)
      client.setTimeout(IDLE_MS, teardown); upstream.setTimeout(IDLE_MS, teardown)
      client.pipe(upstream).pipe(client)
    } catch (err) {
      client.destroy()
    }
  })
  await new Promise<void>((res, rej) => {
    server.once('error', rej)
    server.listen(hostPort, '127.0.0.1', () => res())
  })
  OPEN.set(`${workerId}:${containerPort}`, server)
}

export async function stop(workerId: string, containerPort: number) {
  const key = `${workerId}:${containerPort}`
  const server = OPEN.get(key)
  if (!server) return
  await new Promise(res => server.close(res))
  OPEN.delete(key)
}

export async function stopAllForWorker(workerId: string) {
  for (const key of OPEN.keys()) {
    if (key.startsWith(`${workerId}:`)) {
      await stop(...key.split(':') as [string, number])
    }
  }
}

On Nexus boot, restore listeners for status='approved' rows of running workers.

4. Host port allocation

Pool 10000–10999 (configurable via env var). Allocate the lowest free port not in the DB. On server.listen error (EADDRINUSE), bump and retry. This handles the case where another host process is holding the port.

5. API routes

Operator routesnexus/src/preview/routes.ts, all guarded by requireUnlocked:

Method Path Body Effect
GET /api/workers/:id/previews list
POST /api/workers/:id/previews {container_port, label?} manual add → pending
POST /api/workers/:id/previews/:previewId/approve allocate host port, start proxy, → approved, returns {host_port, url}
POST /api/workers/:id/previews/:previewId/revoke stop proxy, → revoked
DELETE /api/workers/:id/previews/:previewId stop + delete row

Agent callback routenexus/src/preview/agent-routes.ts, bridge-network-only, no requireUnlocked:

Method Path Body Effect
POST /api/agent/previews {worker_id, container_port, label?} look up worker, inspect container, compare its bridge IP to req.ip; on match, upsert pending row (idempotent on (worker_id, container_port)); on mismatch, 403

Both files registered in nexus/src/server.ts next to the existing route families.

6. Worker lifecycle integration

  • Extend removeWorker in nexus/src/workers/service.ts to await tcpProxy.stopAllForWorker(workerId) before container removal.
  • Hook the existing worker health-poll so proxies for non-running workers are auto-revoked (set rows back to pending rather than deleting them, so the next start can re-approve fast).

7. UI

New "Previews" sub-section per workspace in nexus/web/src/views/workers.ts, sibling to the sessions mount:

  • Compact list of rows, each showing: status badge, container port, label.
  • Approved rows: host URL as a target="_blank" link + a "Copy" button + a "Revoke" button.
  • Pending rows: "Approve" + "Remove" buttons.
  • Below the list: a small form (port + label + "Add") for manual ports (skipping the agent callback).
  • Updates on the existing poll cadence (no new SSE/WS).

Out of scope

  • Iframe-embedded viewer. Operator opens previews in a new tab via the host URL. Skipping iframes avoids X-Frame-Options/CSP fights with dev servers and keeps v1 small.
  • Reverse-proxy / subdomain URLs (Codespaces-style <port>-<workspace>.example.com). Not needed for single-operator localhost flow.
  • HTTPS termination. Dev servers run plain HTTP and the listener is loopback-only.
  • UDP forwarding. Stay at TCP.
  • Auto-approve mode. Always operator-gated.

Verification

  • Spawn a worker, docker exec into it, run python3 -m http.server 8000. Manually add port 8000 via the UI form. Approve. Get a host URL like http://127.0.0.1:10000. Open in new tab → directory listing.
  • Repeat using notify-preview 8000 "test" from inside the container. Confirm a pending row appears in the UI; approve; URL works.
  • Repeat with Vite (npx vite). Confirm Vite responds Blocked request. This host is not allowed. Set server.allowedHosts: ['127.0.0.1'] in vite.config.js. Re-open URL — works.
  • Confirm Vite HMR survives an edit (WebSocket upgrade through the raw TCP proxy).
  • Stop the worker (docker stop). Listener disappears within one health-poll cycle. Revoke the row afterwards — returns cleanly (404 or "already gone").
  • Multi-port: declare two ports on the same worker. Both work concurrently on different host ports.
  • Negative: curl http://localhost:3001/api/agent/previews from the host (i.e. not from a bridge IP) returns 403.
  • Negative: agent POSTs with the wrong worker_id (a different worker's id) returns 403 because the IP-vs-container check fails.

Files touched

## Motivation When an agent runs `npm run dev` inside a worker container, the operator currently has no way to open the dev server in a browser. This issue adds: 1. A **worker-reachable Nexus API** so agents can self-register "I have a dev server on port X" — already-on-bridge containers POST to Nexus via Docker DNS, no host port published. 2. A **host-side TCP listener** that pipes traffic to the container's bridge IP on the agent-declared port. Started only after the operator clicks Approve. 3. **Operator-facing UI** to list pending/approved ports, approve/revoke them, and copy the host URL to open in a new browser tab. Companion to #2 (Playwright MCP for agents). This issue is purely the operator-side preview channel; the agent's own browser access is independent. ## Threat model - **Single-operator design** carries over. Host listener binds `127.0.0.1` only — anyone with loopback on the Nexus host already has equivalent access to the UI. - **Two-step gate**: agent declares port → row is `pending` → operator approves → listener opens. No auto-forwarding ever. - **Agent callback endpoints (`/api/agent/*`) are internal-only**. They are reachable from the shared Docker bridge but are **never published** on a host port. Primary defense: don't expose them outside the bridge. Defense in depth: a Fastify route-prefix that returns 403 unless `req.ip` is in a Docker bridge subnet. - **Idle timeouts** on each proxied connection (5 min default) prevent connection leaks. The TCP proxy doesn't parse HTTP so Slowloris isn't a relevant attack surface. - **Listeners torn down on container `die`**. Hook into the existing health-poll rather than subscribing to Docker events (avoids needing `EVENTS=1` on docker-socket-proxy). ## Verified facts (corrections to the original proposal) - `NetworkSettings.IPAddress` (top-level) is **only** populated on the legacy default bridge. Compose-managed networks need `NetworkSettings.Networks[<networkName>].IPAddress`. Use `Object.values(info.NetworkSettings.Networks)[0].IPAddress`. **Don't cache** the result — bridge IPs reassign on restart. - The existing docker-socket-proxy config in [docker-compose.yml#L10](docker-compose.yml#L10) already sets `CONTAINERS=1`, which **does** cover `GET /containers/{id}/json`. **No new socket-proxy permissions are needed.** `NETWORKS=1` is only for the separate `/networks/*` endpoints, which we don't use. - Raw `net.createServer` + `socket.pipe()` is fully transparent at L4 — WebSocket upgrades, HMR, and long-lived connections all work. Don't reach for `http-proxy` / `http-proxy-middleware`; they parse HTTP and break Next.js HMR. - `dockerode` 5.0.0 is current and `docker.getContainer(id).inspect()` returns the full `NetworkSettings` we need. - **Vite caveat**: Vite ≥ 6 will reject requests with an unexpected Host header (`Blocked request. This host is not allowed.`) unless `server.allowedHosts` lists it. Operator needs to set `server.allowedHosts: ['127.0.0.1']` in their `vite.config.js`. Next.js dev has no such issue. ## Suggested implementation ### 1. Worker-reachable Nexus API Nexus and worker containers both attach to the `default` bridge ([docker-compose.yml#L46-L48](docker-compose.yml#L46-L48)), so workers can already reach `agent-nexus:3001` via Docker DNS. Two things to add: - **Bind Fastify on `0.0.0.0:3001`** (or add a second listener bound to the bridge IP). The host publish in compose stays `127.0.0.1:3001:3001` — **don't** publish a second host port. This keeps the agent endpoints reachable from inside the bridge but not from outside. - **Agent-callback route prefix `/api/agent/*`**, no `requireUnlocked` guard. Add a small `preHandler` that returns 403 unless `req.ip` is in a private/bridge range (e.g. `172.16.0.0/12`, `192.168.0.0/16`). Defense in depth — primary defense is the network policy above. A small shell helper baked into the worker image (`worker/notify-preview.sh` → `/usr/local/bin/notify-preview`): ```bash #!/bin/bash # Usage: notify-preview <port> [label] PORT=$1 LABEL=${2:-"dev server"} curl -sf -X POST "http://agent-nexus:3001/api/agent/previews" \ -H "Content-Type: application/json" \ -d "{\"worker_id\":\"$WORKER_ID\",\"container_port\":$PORT,\"label\":\"$LABEL\"}" \ && echo "Preview registered: port $PORT" \ || echo "Failed to notify Nexus (non-fatal)" ``` The worker spawn in [nexus/src/workers/service.ts#L139](nexus/src/workers/service.ts#L139) needs to add `WORKER_ID=<id>` to the container's `Env:` so the helper has it. Not a secret — labels and `inspect` already expose it. ### 2. Schema New migration `nexus/migrations/000N_preview_ports.ts`: ```ts await db.schema .createTable('preview_ports') .addColumn('id', 'text', c => c.primaryKey()) .addColumn('worker_id', 'text', c => c.notNull().references('workers.id').onDelete('cascade')) .addColumn('container_port', 'integer', c => c.notNull()) .addColumn('host_port', 'integer') .addColumn('status', 'text', c => c.notNull()) // 'pending' | 'approved' | 'revoked' .addColumn('label', 'text') .addColumn('created_at', 'integer', c => c.notNull()) .addColumn('approved_at', 'integer') .execute() await db.schema .createIndex('preview_ports_worker_port_uq') .on('preview_ports').columns(['worker_id', 'container_port']).unique() .execute() await db.schema .createIndex('preview_ports_host_port_uq') .on('preview_ports').column('host_port').unique() .where('host_port', 'is not', null) .execute() ``` ### 3. TCP proxy service New `nexus/src/preview/tcp-proxy.ts`: ```ts import net from 'node:net' import type { Container } from 'dockerode' const OPEN: Map<string, net.Server> = new Map() // key = `${workerId}:${port}` const IDLE_MS = 5 * 60 * 1000 async function resolveContainerIp(container: Container): Promise<string> { const info = await container.inspect() const network = Object.values(info.NetworkSettings.Networks)[0] if (!network?.IPAddress) throw new Error('no bridge IP') return network.IPAddress } export async function start(workerId: string, containerId: string, containerPort: number, hostPort: number) { const server = net.createServer({ allowHalfOpen: false }, async (client) => { try { const ip = await resolveContainerIp(docker.getContainer(containerId)) const upstream = net.connect(containerPort, ip) const teardown = () => { client.destroy(); upstream.destroy() } client.on('error', teardown); upstream.on('error', teardown) client.setTimeout(IDLE_MS, teardown); upstream.setTimeout(IDLE_MS, teardown) client.pipe(upstream).pipe(client) } catch (err) { client.destroy() } }) await new Promise<void>((res, rej) => { server.once('error', rej) server.listen(hostPort, '127.0.0.1', () => res()) }) OPEN.set(`${workerId}:${containerPort}`, server) } export async function stop(workerId: string, containerPort: number) { const key = `${workerId}:${containerPort}` const server = OPEN.get(key) if (!server) return await new Promise(res => server.close(res)) OPEN.delete(key) } export async function stopAllForWorker(workerId: string) { for (const key of OPEN.keys()) { if (key.startsWith(`${workerId}:`)) { await stop(...key.split(':') as [string, number]) } } } ``` On Nexus boot, restore listeners for `status='approved'` rows of running workers. ### 4. Host port allocation Pool 10000–10999 (configurable via env var). Allocate the lowest free port not in the DB. On `server.listen` error (`EADDRINUSE`), bump and retry. This handles the case where another host process is holding the port. ### 5. API routes **Operator routes** — `nexus/src/preview/routes.ts`, all guarded by `requireUnlocked`: | Method | Path | Body | Effect | |---|---|---|---| | `GET` | `/api/workers/:id/previews` | — | list | | `POST` | `/api/workers/:id/previews` | `{container_port, label?}` | manual add → `pending` | | `POST` | `/api/workers/:id/previews/:previewId/approve` | — | allocate host port, start proxy, → `approved`, returns `{host_port, url}` | | `POST` | `/api/workers/:id/previews/:previewId/revoke` | — | stop proxy, → `revoked` | | `DELETE` | `/api/workers/:id/previews/:previewId` | — | stop + delete row | **Agent callback route** — `nexus/src/preview/agent-routes.ts`, bridge-network-only, no `requireUnlocked`: | Method | Path | Body | Effect | |---|---|---|---| | `POST` | `/api/agent/previews` | `{worker_id, container_port, label?}` | look up worker, inspect container, compare its bridge IP to `req.ip`; on match, upsert `pending` row (idempotent on `(worker_id, container_port)`); on mismatch, 403 | Both files registered in [nexus/src/server.ts](nexus/src/server.ts) next to the existing route families. ### 6. Worker lifecycle integration - Extend `removeWorker` in [nexus/src/workers/service.ts](nexus/src/workers/service.ts) to `await tcpProxy.stopAllForWorker(workerId)` before container removal. - Hook the existing worker health-poll so proxies for non-running workers are auto-revoked (set rows back to `pending` rather than deleting them, so the next start can re-approve fast). ### 7. UI New "Previews" sub-section per workspace in [nexus/web/src/views/workers.ts](nexus/web/src/views/workers.ts), sibling to the sessions mount: - Compact list of rows, each showing: status badge, container port, label. - Approved rows: host URL as a `target="_blank"` link + a "Copy" button + a "Revoke" button. - Pending rows: "Approve" + "Remove" buttons. - Below the list: a small form (`port` + `label` + "Add") for manual ports (skipping the agent callback). - Updates on the existing poll cadence (no new SSE/WS). ## Out of scope - **Iframe-embedded viewer.** Operator opens previews in a new tab via the host URL. Skipping iframes avoids X-Frame-Options/CSP fights with dev servers and keeps v1 small. - **Reverse-proxy / subdomain URLs** (Codespaces-style `<port>-<workspace>.example.com`). Not needed for single-operator localhost flow. - **HTTPS termination.** Dev servers run plain HTTP and the listener is loopback-only. - **UDP forwarding.** Stay at TCP. - **Auto-approve mode.** Always operator-gated. ## Verification - Spawn a worker, `docker exec` into it, run `python3 -m http.server 8000`. Manually add port 8000 via the UI form. Approve. Get a host URL like `http://127.0.0.1:10000`. Open in new tab → directory listing. - Repeat using `notify-preview 8000 "test"` from inside the container. Confirm a pending row appears in the UI; approve; URL works. - Repeat with Vite (`npx vite`). Confirm Vite responds `Blocked request. This host is not allowed.` Set `server.allowedHosts: ['127.0.0.1']` in `vite.config.js`. Re-open URL — works. - Confirm Vite HMR survives an edit (WebSocket upgrade through the raw TCP proxy). - Stop the worker (`docker stop`). Listener disappears within one health-poll cycle. Revoke the row afterwards — returns cleanly (404 or "already gone"). - Multi-port: declare two ports on the same worker. Both work concurrently on different host ports. - Negative: `curl http://localhost:3001/api/agent/previews` from the host (i.e. not from a bridge IP) returns 403. - Negative: agent POSTs with the wrong `worker_id` (a different worker's id) returns 403 because the IP-vs-container check fails. ## Files touched - New: `nexus/src/preview/tcp-proxy.ts`, `nexus/src/preview/routes.ts`, `nexus/src/preview/agent-routes.ts`, `nexus/migrations/000N_preview_ports.ts`, `worker/notify-preview.sh`. - Modified: [nexus/src/server.ts](nexus/src/server.ts), [nexus/src/workers/service.ts](nexus/src/workers/service.ts) (Env + removeWorker hook), [nexus/web/src/views/workers.ts](nexus/web/src/views/workers.ts), [worker/Dockerfile](worker/Dockerfile) (install notify-preview helper). - Unchanged: [docker-compose.yml](docker-compose.yml) — the docker-socket-proxy allowlist and the host port mapping are both already correct.
lz closed this issue 2026-05-30 10:49:28 +02:00
Author
Owner

Closed by PR #21

Closed by PR #21
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lz/agent-nexus#3
No description provided.