Skip to content

feat(server): rotate ENCRYPTION_KEY without losing unexpired secrets - #170

Merged
dillonstreator merged 2 commits into
mainfrom
feat/encryption-key-rotation
Nov 13, 2026
Merged

dillonstreator merged 2 commits into
mainfrom
feat/encryption-key-rotation

Conversation

@dillonstreator

@dillonstreator dillonstreator commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

Summary

Today, changing ENCRYPTION_KEY makes every unexpired secret unreadable. This adds a keyring so the key can be rotated without losing data, following the keyset pattern used by Google Tink and Rails Active Record Encryption: one primary key for writes, previous keys for reads only, and a key id stored with each value.

  • Core (machine): createKeyring, encryptWithKeyring, decryptWithKeyring, digestWithKeyring and digestCandidates. New formats are m2:<kid>: (AES-256-GCM, with the header bound as associated data so a relabelled key id fails) and hm2:<kid>: (HMAC-SHA-256). The key id is derived from the key, so operators never manage ids. Existing m1/hm1/legacy PBKDF2 values still read under any key in the ring.
  • Server: new ENCRYPTION_PREVIOUS_KEYS (comma-separated, read-only). The vault and the BullMQ webhook queue use the keyring. On startup the server logs how many previous keys it loaded. Key ids are never logged or printed, because they are derived from the keys and would let anyone reading the output check guesses of a weak key offline.
  • rotate-keys command (pnpm --filter @crypt.fyi/server keys:rotate, or node packages/server/dist/rotate-keys.js in Docker, with optional --dry-run): re-encrypts IP restrictions, webhook URLs and delete tokens with the primary key. Each write is a compare-and-set that keeps the entry's TTL, so it never overwrites a concurrent read, decrement or burn. It then reports when previous keys can be removed (previousKeysNeededUntil, safeToRemovePreviousKeys). Only counts are printed.
  • Safety fix: when an entry's key isn't loaded, reads now fail without using up a failed attempt, so a misconfigured keyring can't burn entries.

Verification-hash digests are one-way HMACs (required by SPECIFICATION §6.2), so they can't be re-keyed without the original hash. Entries created before a rotation keep needing the old key until they expire, at most the max TTL. The runbook and the command's report cover this.

Docs: README "Rotating ENCRYPTION_KEY" runbook, SPECIFICATION §6.2 and §7.2.1, docker-compose.yaml, .env-example, and a changeset.

Changeset

This is a minor bump. Because the packages are versioned together, everything moves to 0.2.0.

Testing

  • Core: 65 tests pass, including new keyring tests (round-trip, key id tagging, relabel rejection, legacy formats, digest candidates, and never accepting a stored digest as h).
  • Server: 27 tests pass, including 11 new rotation tests against Redis (reading with previous keys, hm1/m1 entries, rewrap preserving TTL and fc, dry run, unreadable entries, no burn when a key is missing).
  • pnpm lint, pnpm typecheck and pnpm format:check pass for the touched packages. Ran the CLI by hand with --dry-run.
  • pnpm test:e2e was not run locally.

Hold a keyring (primary ENCRYPTION_KEY plus read-only ENCRYPTION_PREVIOUS_KEYS)
and tag every ciphertext and digest at rest with the id of the key that wrote
it. Add a rotate-keys command that re-encrypts vault metadata with the primary
key and reports when previous keys can be removed. Reads of entries whose key
is not loaded now fail without consuming a failed attempt.
- Catch decrypt failures in the webhook 'failed' listener so a removed key
  can't cause an unhandled rejection.
- Skip the per-key PBKDF2 legacy fallback for values that can't be base64
  ciphertext (plaintext IP restrictions).
- Stop logging and printing key ids, which act as an offline verifier for weak
  keys; report counts instead.
- Don't count hm1/m1 values on the primary key as depending on previous keys.
- Dedupe SCAN results and page through queued webhook jobs.
- Cache per-key HMAC keys in the keyring; share format prefix constants;
  simplify field accessors and drop a leftover alias.
@dillonstreator
dillonstreator merged commit 4d0c330 into main Nov 13, 2026
8 checks passed
@dillonstreator
dillonstreator deleted the feat/encryption-key-rotation branch November 13, 2026 02:27
@github-actions github-actions Bot mentioned this pull request Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant