Envelope-encrypt the vault: one DEK, per-principal wrapped keys

lz commented

2026-09-12 19:35:14 +02:00

Owner

Closes the blocker on #140.

What changes

Before, the AES key sealing connections.encrypted_token and repo_env_vars.encrypted_value was the Argon2id output of the master passphrase. Key and credential were 1:1, which made rotation O(rows) and a second credential impossible.

Now one random 32-byte DEK seals all data, and each principal holds a copy of that DEK wrapped under a KEK derived from its own credential:

KEK     = Argon2id(passphrase, salt, params)   # per principal
wrapped = seal(KEK, DEK)                       # stored per principal
DEK     = open(KEK, wrapped)                   # unlock
<data>  = seal(DEK, plaintext)                 # every sealed column

seal/open and deriveKey are untouched — only what key is handed to them changes. New table vault_keys (migration 0020) holds one row per principal; exactly one exists today, ('master','master'). Multi-user is now a row insert, which is the entire point.

Consequences:

Rotation is O(1). It re-wraps the DEK and touches no data column; the ciphertext is asserted byte-identical.
The verifier is retired, not replaced. A wrong passphrase now falls out of the AES-GCM tag failure on wrapped_dek. A second thing that can disagree with the first is gone.
locals.masterKey is now the DEK, and every one of its ~40 consumers is opaque to the change — they only ever hand a Buffer to seal/open.

Design: docs/superpowers/specs/2026-09-11-vault-envelope-encryption-design.md. Mechanics recorded as AGENTS.md fact #27.

The legacy upgrade

It cannot be a migration — migrations run at boot, and the key only exists once the operator types the passphrase. So it runs lazily on the first successful unlock, re-sealing both columns KEK→DEK, inserting the principal with the existing salt/params, and deleting meta.verifier, all in one transaction. Measured: unwrap that transaction and a throw partway through leaves rows sealed under a DEK that was never persisted, while the surviving verifier still reports the install as legacy — neither key opens them.

⚠️ The upgrade is one-way. Once it has run, reverting to the pre-envelope image leaves an install no passphrase opens through the UI. Back up nexus-data before deploying.

Review

SDD per-task reviews, a whole-feature review, and a final /pr-review pass (six agents). The last pass found three guards that could not fail, each fixed and each proven to fail before the fix:

The rotation route's defensive Buffer.from() copy had no test. Removing it left all 69 tests green while seating the operator's new session on 32 zero bytes — destroyAll() zeroizes session keys and SessionStore.create stores by reference. Every secret would fail to open immediately after a rotation, silently.
Folding unlockMasterPassphrase's fall-through re-read into its catch left all 70 tests green while reporting a correct passphrase as wrong for the race shape that returns null instead of throwing.
sealed-columns.test.ts is satisfied by a bare select() naming the column — now stated honestly in the test rather than overclaimed.

It also fixed two real defects: setupMasterPassphrase was check-then-upsert on an unauthenticated route, so a concurrent setup could silently replace the winner's wrapper (writePrincipal is now split into insertPrincipal / upsertPrincipal, making the rule structural and deleting the ten-line comment that used to defend it); and the race catch called readPrincipal, whose JSON.parse could throw a SyntaxError that replaced the damage error being judged.

Six false comments were corrected — three written on this branch and falsified by a later commit on the same branch, which is the class per-task review structurally cannot see. Including a wrong security claim (a shared salt does not make one principal's KEK derivable from another's parameters; it amortises one Argon2id precomputation), also corrected in the design doc.

Comment density: vault-envelope.ts 29%→22%, migration 31%→18%, sealed-columns.test.ts 43%→34%, vault-store.ts 38%→30%. Most of what went was a third copy of arguments already in the design doc and fact #27.

Safety check on the live instance

There was a real window where rotation re-sealed connections but not repo_env_vars: c8aa70b (2026-07-16 19:37Z) → c472266 (2026-07-17 10:13Z), both on main. A row left dead by that window would, after this branch, make every unlock throw an uncaught 500 — permanent lockout, recoverable only by hand-editing SQLite.

Checked before opening this PR: the instance is clear. One repo_env_vars row, created 2026-07-17T17:24Z — 7.2 hours after the fix, outside the window.

Follow-ups filed, not fixed here

#142 (one unopenable row bricks unlock — the throw is right, the uncaught 500 is not), #143 (a lost vault_keys row routes to /setup, concealing the loss), #144 (rotation reports failure for a rotation that succeeded), #145 (dead ctEqual, PrincipalKind union).

Gates

1532 tests passing (146 files), typecheck 0 errors across 4944 files, lint clean.

Closes the blocker on #140. ## What changes Before, the AES key sealing `connections.encrypted_token` and `repo_env_vars.encrypted_value` **was** the Argon2id output of the master passphrase. Key and credential were 1:1, which made rotation O(rows) and a second credential impossible. Now one random 32-byte **DEK** seals all data, and each principal holds a copy of that DEK wrapped under a **KEK** derived from its own credential: ``` KEK = Argon2id(passphrase, salt, params) # per principal wrapped = seal(KEK, DEK) # stored per principal DEK = open(KEK, wrapped) # unlock <data> = seal(DEK, plaintext) # every sealed column ``` `seal`/`open` and `deriveKey` are untouched — only *what key is handed to them* changes. New table `vault_keys` (migration 0020) holds one row per principal; exactly one exists today, `('master','master')`. **Multi-user is now a row insert**, which is the entire point. Consequences: - **Rotation is O(1).** It re-wraps the DEK and touches no data column; the ciphertext is asserted byte-identical. - **The verifier is retired, not replaced.** A wrong passphrase now falls out of the AES-GCM tag failure on `wrapped_dek`. A second thing that can disagree with the first is gone. - **`locals.masterKey` is now the DEK**, and every one of its ~40 consumers is opaque to the change — they only ever hand a `Buffer` to `seal`/`open`. Design: `docs/superpowers/specs/2026-09-11-vault-envelope-encryption-design.md`. Mechanics recorded as AGENTS.md fact #27. ## The legacy upgrade It **cannot be a migration** — migrations run at boot, and the key only exists once the operator types the passphrase. So it runs lazily on the first successful unlock, re-sealing both columns KEK→DEK, inserting the principal with the existing salt/params, and deleting `meta.verifier`, **all in one transaction**. Measured: unwrap that transaction and a throw partway through leaves rows sealed under a DEK that was never persisted, while the surviving verifier still reports the install as legacy — neither key opens them. ⚠️ **The upgrade is one-way.** Once it has run, reverting to the pre-envelope image leaves an install no passphrase opens through the UI. **Back up `nexus-data` before deploying.** ## Review SDD per-task reviews, a whole-feature review, and a final `/pr-review` pass (six agents). The last pass found three guards that could not fail, each fixed and each proven to fail before the fix: - The rotation route's defensive `Buffer.from()` copy had **no test**. Removing it left all 69 tests green while seating the operator's new session on **32 zero bytes** — `destroyAll()` zeroizes session keys and `SessionStore.create` stores by reference. Every secret would fail to open immediately after a rotation, silently. - Folding `unlockMasterPassphrase`'s fall-through re-read into its catch left all 70 tests green while reporting a **correct passphrase as wrong** for the race shape that returns null instead of throwing. - `sealed-columns.test.ts` is satisfied by a bare `select()` naming the column — now stated honestly in the test rather than overclaimed. It also fixed two real defects: `setupMasterPassphrase` was check-then-**upsert** on an unauthenticated route, so a concurrent setup could silently replace the winner's wrapper (`writePrincipal` is now split into `insertPrincipal` / `upsertPrincipal`, making the rule structural and deleting the ten-line comment that used to defend it); and the race catch called `readPrincipal`, whose `JSON.parse` could throw a `SyntaxError` that **replaced** the damage error being judged. Six false comments were corrected — three written on this branch and falsified by a *later commit on the same branch*, which is the class per-task review structurally cannot see. Including a wrong security claim (a shared salt does not make one principal's KEK derivable from another's parameters; it amortises one Argon2id precomputation), also corrected in the design doc. Comment density: `vault-envelope.ts` 29%→22%, migration 31%→18%, `sealed-columns.test.ts` 43%→34%, `vault-store.ts` 38%→30%. Most of what went was a third copy of arguments already in the design doc and fact #27. ## Safety check on the live instance There was a real window where rotation re-sealed `connections` but not `repo_env_vars`: `c8aa70b` (2026-07-16 19:37Z) → `c472266` (2026-07-17 10:13Z), both on `main`. A row left dead by that window would, after this branch, make **every** unlock throw an uncaught 500 — permanent lockout, recoverable only by hand-editing SQLite. **Checked before opening this PR: the instance is clear.** One `repo_env_vars` row, created 2026-07-17T17:24Z — 7.2 hours after the fix, outside the window. ## Follow-ups filed, not fixed here #142 (one unopenable row bricks unlock — the throw is right, the uncaught 500 is not), #143 (a lost `vault_keys` row routes to `/setup`, concealing the loss), #144 (rotation reports failure for a rotation that succeeded), #145 (dead `ctEqual`, `PrincipalKind` union). ## Gates 1532 tests passing (146 files), typecheck 0 errors across 4944 files, lint clean.

lz added 25 commits

2026-09-12 19:35:15 +02:00

docs: spec envelope encryption for the vault 017eaf1a67

The vault key is currently Argon2id(passphrase), so key and credential are
1:1. Multi-user breaks that identity permanently, rotation is O(rows) because
of it, and there is no way to grant a non-human principal vault access without
handing out the passphrase.

One DEK seals the data; each principal holds a wrapped copy. Rotation becomes
re-wrapping one blob, which retires the bug class sealed-columns.test.ts was
built to police rather than continuing to police it.

Unblocks #140.

docs: implementation plan for vault envelope encryption 1c2fed4083

docs: note svelte-kit sync in the plan's prerequisites fbb7d09f5f

feat(vault): add vault_keys table for per-principal wrapped keys 4334480f14

docs(schema): name the future principal_kind vocabulary at the interface site c878fe16ff

feat(vault): add DEK generation, wrapping, and principal row IO ab185c9e27

docs(vault): make unwrapDek's comment describe what it actually does 5b27ddecd7

docs: drop a false test citation from the plan, frame the Task 7 guard as a backstop ba0a7ebb29

The upgrade comment cited sealed-columns.test.ts as policing its reasoning,
which only becomes true at Task 7. A comment that is false for four commits
is not acceptable, so the citation goes and the argument stays.

Task 7's check is kept despite that argument saying it is unnecessary: if the
argument holds the guard costs a no-op loop, and if it stops holding the cost
is an unreadable vault.

feat(vault): upgrade a legacy vault to envelope encryption on unlock 3e7839339f

docs(vault): drop the test citation from the upgrade comment d784bf2822

Cited in either polarity it goes stale: Task 7 repoints sealed-columns.test.ts
at this file, which makes 'does NOT police this' false. The argument stands on
its own; the accurate citation lands with the test that earns it.

docs(vault): record the measured cost of unwrapping the upgrade transaction d30a4f39f0

docs: handle the concurrent-unlock race in Task 4 6183125f40

Review measured three losing shapes on the one call that performs the upgrade:
a UNIQUE violation, a decrypt error byte-identical to a corrupt data row, and
a plain null that reports a correct passphrase as wrong. The third never
throws, so a catch alone cannot cover it.

The fix belongs at the unlock site rather than inside upgradeLegacyVault: it
needs readPrincipal, which already runs there, and the null shape is only
visible there.

fix(vault): close two vacuous guards in the legacy upgrade tests 41ae25db10

The suite could not see the upgrade wiping meta.kdf_salt (isSetupComplete
reads it, so the operator would be routed to /setup with an intact vault),
and nothing pinned the vault_keys insert as a plain insert rather than an
upsert — harmonizing it with writePrincipal would replace a live wrapper
with a DEK the rows were never sealed under.

Also: cover both re-seal loops with two rows each, add a rollback test whose
throw lands in the second loop so the first has certainly written, and zero
the DEK on the throw path.

docs: record that Tasks 4, 6 and 7 must land together 64b557e580

The plan sequenced unlock before rotation. In the gap, a successful rotation
locks the operator out under the new passphrase while the old one still
authenticates and opens nothing — found by probing the half-applied state,
not by the suite, which never unlocks after rotating.

docs: separate the measured race shapes from the reasoned one 7fd277cad6

Two of the three were run against a test db with their error strings quoted;
the third was derived by reading the code. One 'Measured:' covering all three
asserts a measurement nobody performed.

feat(vault): unlock and rotate through the envelope b6daea50ed

Unlock prefers a vault_keys principal and upgrades a legacy install in place.
Rotation re-wraps that principal's DEK instead of re-sealing every row.

These land together because they cannot be separated: once unlock prefers the
principal, the old rotation writes a fresh legacy verifier that unlock never
consults, which locks the operator out under the new passphrase while the old
one still authenticates and opens nothing. sealed-columns.test.ts comes along
for the same reason — it asserts rotation re-seals, and its replacement
asserts the opposite.

docs: list the Task 4/6 scaffolding that Task 5 must remove a4a844d7ae

An unlock added to the salt-rotation test becomes a no-op once setup writes the
principal, and its comment becomes false. Nothing fails either way, which is
why it needs a list entry rather than a reviewer noticing.

fix(vault): eight review findings on the envelope rewire e7f4f3582e

Rotation's unlock moves inside the try, so a corrupt row on a legacy install
returns {ok:false, reason:'error'} with the real message instead of rejecting
into SvelteKit's raw Internal Error, which _handleRotate does not catch.

RotateResult.newKey becomes .key: it is the same DEK the caller already held,
and the old name invited a caller to assume stale sessions were already dead
by virtue of the key changing. store.destroyAll() carries that alone.

The rotate docstring regains the residual-risk disclosure the plan mandated —
a DEK exfiltrated from memory survives a rotation. The race-swallow branch
gains a logger.warn, the only evidence it can leave; no test reaches it.

Two comments stated things that were not true of the code beneath them, and
the legacy on-disk shape is now declared once in vault-envelope.ts and read by
both hand-built fixtures.

test(vault): cover the race the legacy upgrade can lose bba8c7e223

The two re-reads in unlockMasterPassphrase had no test: replacing the whole
catch with a bare `throw err` left all 65 tests green. They are reachable
through the db parameter, which is a real seam — a vault_keys read that is
empty and then is not IS a concurrent commit, as far as this function can
observe.

The db is a pass-through proxy over a real openTestDb(), with exactly one
side effect: the first empty vault_keys read commits a genuine principal row
before returning. Schema, queries and the throw stay real, so the test asserts
against Kysely rather than against a reimplementation of it.

It pins both re-reads — deleting either now fails it — and asserts the loser
ends up with the winner's DEK, not merely that nothing threw.

feat(vault): seal new installs under a DEK and retire the verifier 7357541e2b

setupMasterPassphrase now writes a vault_keys master principal directly
instead of the legacy salt/params/verifier meta rows, so a brand-new
install no longer takes a detour through upgradeLegacyVault on its
first unlock. isSetupComplete checks the principal first and falls
back to the legacy meta row so an un-upgraded pre-envelope install
still reads as set up.

fix(vault): guard isSetupComplete's legacy fallback and share seedLegacyVault 208149075e

Without the legacy-meta fallback, an un-upgraded pre-envelope install
reads as "not set up": /api/state would route the operator to /setup,
which would write a fresh principal with a brand-new DEK while every
row is still sealed under the old passphrase's KEK, silently
destroying the vault. Two tests pin this: the predicate itself, and
that setupMasterPassphrase refuses to run over a legacy vault.

Also collapses the three drifted copies of seedLegacyVault (vault-
envelope.test.ts, vault-store.test.ts, vault-store.rotate.test.ts)
into one shared helper in testing/db.ts, built from vault-envelope.ts's
exported META_*/VERIFIER_PLAINTEXT constants.

fix(vault): correct three stale comments, re-test the upgrade path e472a0bd8f

Final-review remainder on the envelope-encryption branch. Three doc comments
were true when written and falsified by a later task:

- unwrapDek cited "the same idiom unlockMasterPassphrase uses for its verifier";
  that function has neither a verifier nor an open() call any more.
- unlockMasterPassphrase's race log claimed the vanished-verifier shape as
  evidence. That shape returns null and never throws, so it cannot reach the
  catch at all — it leaves via the fall-through re-read.
- The upgrade test justified keeping meta.kdf_salt with an isSetupComplete
  branch that no longer reaches it post-upgrade.

Both spec-mandated upgrade tests ran on setupMasterPassphrase fixtures, which
since the setup rewrite have no upgrade to perform. Replaced with one that seeds
a legacy vault, unlocks it, and asserts the row re-seals under the returned DEK
and that a second unlock neither changes the DEK nor rewrites the wrapper.
Mutation-proved: stubbing upgradeLegacyVault to null turns it red.

isSetupComplete now reads existence only. It went through readPrincipal, which
JSON.parses kdf_params, on the unauthenticated /api/state — a corrupt blob threw
there, 500ing the phase machine with no route back to /unlock.

Also: the rotation modal states that rotating leaves the data key unchanged, so
an operator rotating on suspicion of a leak does not read remediation into it
that is not there; two stale test names and one pre-envelope leftover removed.

docs: record the envelope vault, and retire three facts it inverted 2fd05e479c

AGENTS.md gains fact #27: the DEK/KEK split with one principal today, why the
legacy upgrade runs on first unlock and cannot be a migration, why wrapped_dek
is not named encrypted_*, and that rotation is O(1) and re-keys nothing.

Three existing facts described the old scheme:

- #20 said rotateMasterPassphrase re-seals repo_env_vars and that
  sealed-columns.test.ts fails without a rotation pass. Both halves are now
  inverted — rotation re-seals nothing, and the test fails if vault-store.ts
  mentions a sealed column at all.
- #22 and #24 each closed on "no pass in rotateMasterPassphrase", describing a
  pass that exists for nothing.
- #25 called SessionData.key the Argon2id-derived vault key. It is the random
  DEK an Argon2id-derived KEK unwraps; the conclusion drawn from it still holds.

README no longer tells the operator the passphrase *is* the vault key, and the
design doc's Risks section states what rotation does not remediate.

fix(vault): close the setup race, stop masking damage, guard three paths 9fb45fc6bb

Review pass over the envelope branch. Three defects, all with guards that
were proven to fail before the fix landed.

1. The rotation route's defensive Buffer copy had no test. Removing it left
   all 69 tests green while seating the operator's new session on 32 zero
   bytes: destroyAll() zeroizes session keys, and SessionStore.create stores
   the buffer by reference. Now pinned by a test that seals a connection
   before the rotation and opens it with the stored session key.

2. setupMasterPassphrase was check-then-upsert. Two concurrent posts to the
   unauthenticated /api/auth/setup both pass isSetupComplete, and the loser
   silently replaced the winner's wrapper. writePrincipal is now split into
   insertPrincipal (fails on conflict) and upsertPrincipal (rotation only),
   which makes the rule structural instead of a comment — and lets the
   upgrade's ten-line "do not add onConflict here" warning go away.

3. The race catch in unlockMasterPassphrase called readPrincipal, which
   JSON.parses kdf_params; a corrupt blob threw a SyntaxError that replaced
   the damage error being judged. It reads through principalExists now.

Also covers two paths nothing exercised: the third race shape, where the
winner deletes the verifier before the loser reads it and no throw ever
happens (folding the fall-through re-read into the catch passed all 70
tests while reporting a correct passphrase as wrong), and a rotation on a
clean legacy install — the real first-run path for an existing operator.

Fixture duplication moved to testing/db.ts (8 hand-built inserts, 5 copies
of the same unopenable ciphertext) and one test subsumed verbatim by its
neighbour is gone.

docs: correct false claims in the envelope comments and AGENTS.md

ci / images (pull_request) Has been cancelled

Details

ci / nexus (pull_request) Has been cancelled

Details

11fc67eab6

Six claims were wrong or unverifiable, three of them written on this branch
and falsified by a later commit on the same branch.

- "every retry dies on the same row" — it dies on a row the crashed upgrade
  already re-sealed, which is a different row from the one that first threw.
- The unlock header called the UNIQUE-violation race shape "measured" while
  an inline comment twenty lines down called it "not reproduced".
- "Reaching here means nothing to upgrade" omitted the path where the catch
  above swallowed a real throw.
- sealed-columns.test.ts described its checks as greps over "a whole FILE";
  they run over comment-stripped source, and the second is a prefix test,
  not a column name. It also now records that a bare select() satisfies the
  first check, which is true and was not stated.
- The migration claimed a shared salt would make one principal's KEK
  derivable from another's parameters. A KEK needs the passphrase; the real
  cost of a shared salt is one amortised Argon2id precomputation. The same
  wrong claim is corrected in the design doc.
- session.ts still called SessionData.key "the Argon2id-derived master key".
  It is the DEK, which this branch changed. Setup.svelte said the passphrase
  "derives" the key that encrypts tokens; it unlocks it.

Fact #24 said api_tokens is migration 0017. It is 0019 — renumbered after
the collision that crash-looped production.

Fact #27 gains two things it did not say: the verifier check runs before the
upgrade transaction opens, not inside it, and the upgrade is one-way, so
reverting to the pre-envelope release after a single unlock leaves an
install no passphrase can open through the UI.

Comment density: vault-envelope.ts 29% -> 22%, the migration 31% -> 18%,
sealed-columns.test.ts 43% -> 34%, vault-store.ts 38% -> 30%. Most of what
went was a third copy of arguments already in the design doc and fact #27.

lz added 1 commit

2026-09-12 19:41:22 +02:00

Merge branch 'main' into feat/mcp

ci / nexus (pull_request) Successful in 9m10s

Details

ci / images (pull_request) Successful in 28m11s

Details

pr-image-cleanup / delete-pr-images (pull_request) Successful in 10s

Details

2f74646c4f

AGENTS.md was the only conflict: both sides added a fact numbered 27. Main's
async-workspace-removal fact landed first and keeps 27; the vault envelope
fact becomes 28, and the four cross-references in facts #20, #22, #24 and
#25 were repointed with it.

Merged result: 1696 tests passing, typecheck 0 errors across 4960 files,
lint clean.