Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/redis-aof-privacy-copy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@crypt.fyi/core': patch
---

Correct the security copy now that Redis persists data with an AOF: secrets reach disk only in encrypted form, and deleted entries can remain until the log is next compacted.
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,45 @@ It runs with `NODE_ENV=production` and requires `REDIS_URL` and `ENCRYPTION_KEY`

The web client image is not published to GHCR yet; build it with `docker compose` as above.

#### Redis persistence

The compose file stores Redis data in the `redis_data` volume using an append-only file (AOF) that is synced to disk before each write is acknowledged, so secrets and queued webhooks survive restarts and crashes. A secret that was burned stays burned, even after a crash. RDB snapshots stay off because restoring an older snapshot could bring a burned secret back. See the [specification](./SPECIFICATION.md) for the requirements.

What this means for operators:

- **The disk holds sensitive data.** The volume contains ciphertext, keyed digests of verification hashes and encrypted delete tokens, but never plaintext or decryption keys. The digests and tokens rely on `ENCRYPTION_KEY`, so keep that key off this disk where you can. Once a secret is deleted, the server never returns it, even if its ciphertext remains in the AOF until Redis next compacts it. Encrypt the disk.
- **Don't back up the volume.** Restoring a backup can bring back burned secrets.
- **Writes are slower** than without persistence, because each one waits for the disk.
- **`docker compose down -v` deletes the volume** and every unexpired secret with it.

To run without persistence, set `--appendonly no` and remove the volume. A Redis restart then drops all unexpired secrets and queued webhooks. If you run Redis any other way (the GHCR image with your own Redis, a managed service, a hosting platform), apply these settings yourself: by default Redis takes snapshots and doesn't use AOF.

##### Upgrading from RDB snapshots

Deployments from before this change store data in a `dump.rdb` snapshot, which Redis ignores once AOF is on. **Upgrading without these steps loses all unexpired secrets**, and the old snapshot stays on disk. Before pulling the new compose file, turn on AOF in the running Redis:

```bash
docker compose exec redis redis-cli config set appendonly yes
```

Wait until this shows `aof_rewrite_in_progress:0` and `aof_last_bgrewrite_status:ok`:

```bash
docker compose exec redis redis-cli info persistence
```

Then pull the new compose file and restart:

```bash
docker compose up -d
```

Once you've confirmed existing secrets still open, delete the old snapshot:

```bash
docker compose exec redis rm /data/dump.rdb
```

### Railway

[![Deploy on Railway](https://railway.com/button.svg)](https://railway.com/deploy/Pmkrsc?referralCode=ToZEjF&utm_medium=integration&utm_source=template&utm_campaign=generic)
Expand Down
45 changes: 36 additions & 9 deletions SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Abstract
o Web Client (Browser-based interface)
o Web Server (Static file server)
o API Server
o Data Store (Redis for ephemeral storage)
o Data Store (currently Redis, holding entries until they are deleted)

### 2.2. Component Interaction Flow

Expand Down Expand Up @@ -272,12 +272,31 @@ Abstract
### 6.2. Data Security

- Data MUST have automatic expiration (TTL: 1 second to 7 days)
- Burn after reading MUST use atomic Redis operations
- Burn after reading MUST use atomic data store operations
- Read count limits MUST NOT exceed 10 reads
- Failed attempt limits MUST be between 1-10 attempts before burning
- Implementation MUST NOT use persistent storage - ephemeral Redis
storage only
- Data MUST be securely deleted upon expiration or burning
- Implementations MAY persist data to disk or keep it in memory only. A
memory-only data store loses all unexpired entries when it restarts
- Persisted data MUST NOT include plaintext secrets, encryption keys, or
passwords
- An entry is deleted when it expires, is burned, exhausts its failed
attempts, or is removed with its delete token. Deletion carries two
separate guarantees:
- Deleted entries MUST NOT be released. Deciding to release an entry
and deleting it MUST be a single atomic operation, so that concurrent
requests cannot race, and once an entry is deleted the server MUST NOT
release it through the API. This MUST also hold after a data store
restart or crash: an acknowledged deletion MUST be durable, and
recovery MUST NOT restore a deleted entry. Backups of persisted data
SHOULD NOT be taken, because restoring one can bring back deleted
entries
- Deleted entries MUST be purged. The data store MUST release any
snapshot, memory, or other reference to a deleted entry as soon as is
possible at scale. Data that remains on disk but cannot be released
through the API is acceptable until the data store is able to purge
it, which SHOULD happen as soon as it can. Implementations SHOULD
encrypt the underlying disk, because purging does not guarantee that
the physical blocks are erased
- The verification hash MUST NOT be stored in plaintext; servers MUST store
a keyed digest (HMAC-SHA-256) bound to the vault ID, so a datastore dump
alone cannot be replayed to read or burn an entry
Expand Down Expand Up @@ -321,9 +340,16 @@ Abstract

### 7.1. Storage Requirements

#### 7.1.1. Redis Configuration
#### 7.1.1. Data Store Configuration

- Implementations MUST use ephemeral storage (Redis recommended)
- Implementations MUST meet the requirements in Section 6.2 through a data
store adapter. Redis is the current adapter; others, such as PostgreSQL
or SQLite, MAY be added if they meet the same requirements
- The Redis adapter, when persisting data, MUST use `appendonly yes` and
`appendfsync always` so that a deletion is on disk before it is
acknowledged. RDB save points MUST be disabled, because restoring an
older snapshot can bring back a deleted entry. Deleted data is purged
when Redis rewrites the append-only file
- TTL per entry MUST be configurable (1 second to 7 days)
- Automatic expiration and cleanup MUST be implemented
- Atomic operations MUST be used for critical functions:
Expand All @@ -335,8 +361,9 @@ Abstract

- Implementations MUST support concurrent access
- Data consistency MUST be maintained using atomic transactions
- Atomic operations SHOULD use Lua scripts to prevent race conditions
- Concurrent read/delete operations MUST be handled properly
- Atomic operations SHOULD use the data store's native mechanism to prevent
race conditions (Lua scripts in Redis)
- Concurrent read/delete operations MUST meet the guarantees in Section 6.2

#### 7.1.3. Performance Requirements

Expand Down
15 changes: 13 additions & 2 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,26 @@ services:
# TTL'd and must never be silently dropped, so use noeviction: once the cap
# is hit, new writes are rejected (surfacing pressure) rather than evicting
# live secrets. Tune REDIS_MAXMEMORY to the host.
#
# Secrets survive restarts through an AOF synced before each write is
# acknowledged (SPECIFICATION.md 6.2), so a burn can't be undone by a crash.
# RDB save points stay off because restoring an older snapshot could bring a
# burned secret back. The low rewrite threshold limits how long deleted
# entries remain in the log on small instances.
command:
- redis-server
- --maxmemory
- ${REDIS_MAXMEMORY:-256mb}
- --maxmemory-policy
- noeviction
- --save
- '900'
- '1'
- ''
- --appendonly
- 'yes'
- --appendfsync
- always
- --auto-aof-rewrite-min-size
- 1mb
- --loglevel
- warning
volumes:
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"format": "prettier --write .",
"format:check": "prettier --check .",
"test:e2e": "pnpm --filter @crypt.fyi/web test:e2e",
"redis": "docker run --rm -p 6379:6379 -v ./redis-data:/data redis:latest redis-server --save 1 1 --loglevel notice",
"redis": "docker run --rm -p 6379:6379 redis:latest redis-server --save \"\" --appendonly no --loglevel notice",
"changeset": "changeset",
"version-packages": "changeset version",
"release": "node ./scripts/release.mjs",
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/i18n/locales/de.ts
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,8 @@ export const de: TranslationKeys = {
tls: 'TLS-Verschlüsselung für alle API-Kommunikation',
expiration: 'Automatischer Datenablauf mit konfigurierbarer Lebensdauer (TTL)',
deletion: 'Sichere Datenlöschung nach Zugriff oder Ablauf',
storage: 'Keine dauerhafte Speicherung sensibler Informationen',
storage:
'Geheimnisse gelangen nur verschlüsselt auf die Festplatte, und gelöschte Einträge können dort bis zur nächsten Log-Komprimierung verbleiben; Klartext und Schlüssel werden nie gespeichert',
},
},
thirdParty: {
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/i18n/locales/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,8 @@ export const en: TranslationKeys = {
tls: 'TLS encryption for all API communications',
expiration: 'Automatic data expiration with configurable Time-To-Live (TTL)',
deletion: 'Secure deletion of data after access or expiration',
storage: 'No persistent storage of sensitive information',
storage:
'Secrets reach disk only in encrypted form and deleted entries can remain there until the log is next compacted; plaintext and keys are never stored',
},
},
thirdParty: {
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/i18n/locales/es.ts
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,8 @@ export const es: TranslationKeys = {
tls: 'Cifrado TLS para todas las comunicaciones API',
expiration: 'Vencimiento automático de datos con tiempo de vida configurable (TTL)',
deletion: 'Eliminación segura de datos después del acceso o vencimiento',
storage: 'Sin almacenamiento persistente de información sensible',
storage:
'Los secretos solo llegan al disco cifrados y las entradas eliminadas pueden permanecer allí hasta la siguiente compactación del registro; el texto plano y las claves nunca se almacenan',
},
},
thirdParty: {
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/i18n/locales/fr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,8 @@ export const fr: TranslationKeys = {
tls: 'Chiffrement TLS pour toutes les communications API',
expiration: 'Expiration automatique des données avec durée de vie configurable (TTL)',
deletion: 'Suppression sécurisée des données après accès ou expiration',
storage: "Pas de stockage persistant d'informations sensibles",
storage:
"Les secrets n'atteignent le disque que chiffrés et les entrées supprimées peuvent y rester jusqu'au prochain compactage du journal ; le texte en clair et les clés ne sont jamais stockés",
},
},
thirdParty: {
Expand Down
3 changes: 2 additions & 1 deletion packages/core/src/i18n/locales/zh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,8 @@ export const zh: TranslationKeys = {
tls: '所有 API 通信都使用 TLS 加密',
expiration: '具有可配置生存时间 (TTL) 的自动数据过期',
deletion: '访问或过期后安全删除数据',
storage: '不持久存储敏感信息',
storage:
'秘密内容仅以加密形式写入磁盘,已删除的条目可能保留到下一次日志压缩;绝不存储明文和密钥',
},
},
thirdParty: {
Expand Down
13 changes: 8 additions & 5 deletions packages/server/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
services:
redis:
image: redis:latest
# Memory only: dev data is disposable. The image snapshots by default, so
# persistence has to be turned off explicitly.
command:
- redis-server
- --save
- ''
- --appendonly
- 'no'
ports:
- '6379:6379'
volumes:
- redis_data:/data
jaeger:
image: jaegertracing/all-in-one:latest
environment:
- COLLECTOR_OTLP_ENABLED=true
ports:
- 16686:16686
- 4318:4318

volumes:
redis_data:
Loading