Skip to content

fix: persist Redis with a synced AOF and update the spec - #162

Merged
dillonstreator merged 5 commits into
mainfrom
fix/redis-memory-only
Nov 6, 2026
Merged

dillonstreator merged 5 commits into
mainfrom
fix/redis-memory-only

Conversation

@dillonstreator

@dillonstreator dillonstreator commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Summary

SPECIFICATION.md 6.2 said "Implementation MUST NOT use persistent storage", but the default Docker deployment snapshotted Redis to a named volume. Rather than dropping persistence, which loses every unexpired secret on any Redis restart, this switches to a durable AOF and changes the spec to match.

Snapshots were the worst of both: a crash lost recent secrets and could also bring back a secret burned since the last snapshot. With appendfsync always, a burn is on disk before the recipient gets the secret.

  • docker-compose.yaml: --appendonly yes, --appendfsync always, --save '', --auto-aof-rewrite-min-size 1mb, and the redis_data volume stays.
  • SPECIFICATION.md 2.1, 6.2, 7.1: replaces "no persistent storage" with two separate, store-agnostic deletion guarantees. A deleted entry MUST NOT be released: release and delete are one atomic operation, including across crash recovery. A deleted entry MUST be purged: the store releases references as soon as is possible at scale, and data still on disk is acceptable until then as long as the API cannot return it. Redis specifics (AOF always, no RDB save points) move under the adapter requirements, leaving room for PostgreSQL or SQLite adapters.
  • security.*.storage copy in all 5 locales: "No persistent storage of sensitive information" is no longer true, so it now says what does reach disk. Translations are mine and could use a native check.
  • README.md: explains persistence, operator responsibilities and upgrade steps.
  • Dev setups (packages/server/docker-compose.yaml, pnpm redis): memory-only, which the spec allows. Dev data is disposable and this avoids slow writes, but dev Redis no longer keeps data across restarts.

⚠️ Upgrade impact

Just pulling this and running docker compose up -d loses all unexpired secrets. Redis ignores dump.rdb once AOF is on, and that snapshot then stays on disk. Run config set appendonly yes on the live Redis before upgrading. The README has the full steps, and they were tested.

Trade-offs

  • Write cost: about 30x lower Redis write throughput in a single-client redis-benchmark SET test on Docker Desktop for Windows: 273 req/s at p50 3.9 ms with always, against 8,065 req/s at p50 0.11 ms with everysec. Real disks will differ.
  • Data on disk: ciphertext, keyed digests of verification hashes and encrypted delete tokens stay on disk by design, never plaintext or keys. The digests and tokens rely on ENCRYPTION_KEY.
  • Log residue: burned ciphertext stays in the AOF until the next compaction. The 1mb threshold shortens that on small instances, but it depends on write volume, not time, so a quiet instance can keep it for a long time.

Testing

Run against the built root stack unless noted:

  • docker compose config is valid for both compose files.
  • Crash: after docker kill (SIGKILL) straight after a burn, the burned secret stayed gone, and the unburned secret survived with its TTL intact.
  • Restart: a clean restart keeps secrets, and the unburned secret returned 200 over the API after the crash.
  • Compaction (standalone Redis, compression off): burned ciphertext count went from 1 to 0 after BGREWRITEAOF, the kept secret stayed, and it survived a further SIGKILL and reload.
  • Upgrade steps (standalone Redis): a key in an old-config volume kept its value and TTL through the steps, and after removing dump.rdb and restarting.
  • Webhooks: the worker processes new jobs after restarts. Those deliveries to example.com were rejected and logged as failed, which is expected for that target.
  • Races (merged stack, AOF on): with 8 concurrent reads, a burn-after-read secret was released exactly once in each of 3 rounds, and a read-count-3 secret exactly 3 times. A deleted secret returned 404 after a SIGKILL and recovery. In the delete test all 8 reads landed after the delete, so a read interleaved with a delete was not exercised; that case rests on the Lua script being the only thing that authorizes a release.
  • packages/server tests: not run locally, because they hardcode localhost:6379, which another process holds here. CI runs them.
  • Core package: typecheck, lint (one warning in a file not changed here) and format:check pass. All 47 tests pass with an explicit --testMatch, because the configured pattern finds no tests from this dot-directory worktree path. CI runs them normally.

Not tested:

  • pnpm redis itself was not run, so its \"\" quoting hasn't been exercised through pnpm's shell. Please run it once.
  • Survival of a job sitting in the webhook queue during a restart wasn't observed directly, because the worker empties the queue too quickly. It relies on the same AOF as the secrets.
  • The Railway deployment.

Not in this PR

  • Railway: the deploy template is defined outside this repo, so its Redis settings may not meet the updated spec.
  • Other copy: the deletion line ("Secure deletion of data…") and the landing page's "Ephemeral by default" are unchanged. They still read as accurate alongside the new line, but deserve a look.

SPECIFICATION.md 6.2 forbids persistent storage, but the default compose
stack snapshotted Redis to a named volume, so ciphertext, verification
hashes and delete tokens could stay on disk after a burn.

Disable RDB and AOF and drop the volume in both compose files and the
`pnpm redis` script. The image snapshots by default, so removing `--save`
alone would not be enough. Document that a Redis restart now drops
unexpired secrets and queued webhooks, and how to remove an old volume.
@dillonstreator dillonstreator added the no-changeset Skip the changeset requirement (docs, tests, CI-only changes) label Nov 6, 2026
Memory-only Redis met the spec but lost every unexpired secret on any
restart. Snapshots kept data across clean restarts, but a crash could bring
back a secret burned since the last snapshot.

Use an AOF synced before each write is acknowledged, so secrets survive
restarts and an acknowledged burn survives a crash. Update the spec to allow
this and forbid recovering from snapshots, and correct the security copy that
claimed no persistent storage. Document the upgrade steps, because Redis
ignores an existing dump.rdb once AOF is on. Dev setups stay memory-only.
@dillonstreator dillonstreator changed the title fix: run Redis memory-only to match the no-persistence spec fix: persist Redis with a synced AOF and update the spec Nov 6, 2026
@dillonstreator dillonstreator removed the no-changeset Skip the changeset requirement (docs, tests, CI-only changes) label Nov 6, 2026
"No persistent storage" conflated two guarantees. A deleted entry must never
be released, with no races and across crash recovery. It must also be purged
as soon as the backing store can manage at scale, and data still on disk is
acceptable until then.

Keep the requirements store-agnostic so other adapters can be added, and move
the Redis settings under the adapter requirements.
@dillonstreator
dillonstreator merged commit c6a41a4 into main Nov 6, 2026
8 checks passed
@dillonstreator
dillonstreator deleted the fix/redis-memory-only branch November 6, 2026 23:36
@github-actions github-actions Bot mentioned this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant