Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/encrypt-verification-hash-at-rest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@crypt.fyi/core': minor
'@crypt.fyi/server': minor
---

Stop storing verification hashes and delete tokens in plaintext in Redis. Hashes are now stored as a keyed digest bound to the vault ID, and delete tokens are encrypted. Existing entries keep working until they expire. Changing `ENCRYPTION_KEY` now makes all unexpired secrets unreadable, not only those with IP restrictions or webhooks.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
API_URL=https://{your-domain-here} ENCRYPTION_KEY=$(openssl rand -base64 32) docker compose up --build
```

`ENCRYPTION_KEY` encrypts IP restrictions and webhook URLs at rest in Redis. Keep it stable across restarts (e.g. in a `.env` file next to `docker-compose.yaml`); changing it makes existing secrets with those options unreadable.
`ENCRYPTION_KEY` protects verification hashes, delete tokens, IP restrictions and webhook URLs at rest in Redis. Keep it stable across restarts (e.g. in a `.env` file next to `docker-compose.yaml`); changing it makes all existing secrets unreadable until they expire.

`API_URL` is read when the web container starts, so changing it only needs a restart (`docker compose up -d`), not a rebuild. It must be a plain `http(s)://host[:port][/path]` URL; the container refuses to start otherwise.

Expand Down
16 changes: 12 additions & 4 deletions SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ Abstract

- MUST handle only encrypted data operations
- MUST NOT receive or process URLs containing decryption keys
- MUST only receive hashed keys for verification (SHA-512)
- MUST only receive hashed keys for verification (see Section 3.2)
- MUST operate independently from web server

This separation ensures that even if the web server logs are
Expand Down Expand Up @@ -168,7 +168,9 @@ Abstract
### 3.2. Hash Functions

Implementations MUST use:
- SHA-512 for key verification hash (key + optional password)
- SHA-512 for key verification hash without a password (key only), and
for legacy links
- Argon2id for key verification hash with a password (key + password)
- SHA3-512 for key derivation in ML-KEM-768-2

### 3.3. Compression
Expand All @@ -184,7 +186,7 @@ Abstract
```json
{
"c": "<base64-encoded-encrypted-content>",
"h": "<sha512-hash-of-key-and-password>",
"h": "<verification-hash-of-key-and-password>",
"m": {
"compression": {
"algorithm": "zlib:pako" | "none"
Expand Down Expand Up @@ -276,13 +278,19 @@ Abstract
- Implementation MUST NOT use persistent storage - ephemeral Redis
storage only
- Data MUST be securely deleted upon expiration or burning
- The verification hash MUST NOT be stored in plaintext; servers MUST store
a keyed digest (HMAC-SHA-256) bound to the vault ID, so a datastore dump
alone cannot be replayed to read or burn an entry
- Delete tokens MUST NOT be stored in plaintext; servers MUST store them
encrypted so they can be returned in webhook payloads

### 6.3. Access Control

- IP address restrictions MAY use CIDR notation
- Implementations MUST NOT allow more than 3 IP restrictions per
vault entry
- Key authentication MUST use SHA-512 hash verification
- Key authentication MUST use the verification hash defined in
Section 3.2

### 6.4. Transport Security

Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ services:
environment:
- REDIS_URL=redis://redis:6379
- PORT=${SERVER_PORT:-4321}
# Encrypts IP restrictions and webhook URLs at rest in Redis.
# Protects verification hashes, delete tokens, IP restrictions and webhook URLs at rest in Redis.
- ENCRYPTION_KEY=${ENCRYPTION_KEY:?ENCRYPTION_KEY is required (e.g. openssl rand -base64 32)}
ports:
- '${SERVER_PORT:-4321}:${SERVER_PORT:-4321}'
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export const readVaultParamsSchema = z.object({
export type ReadVaultParams = z.infer<typeof readVaultParamsSchema>;

export const readVaultQuerySchema = z.object({
h: z.string().describe('sha512 hash of the encryption key + optional password'),
h: z.string().describe('verification hash of the encryption key + optional password'),
});
export type ReadVaultQuery = z.infer<typeof readVaultQuerySchema>;

Expand Down
30 changes: 29 additions & 1 deletion packages/core/src/encryption/machine.test.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,41 @@
import { encrypt, decrypt } from './machine';
import { encrypt, decrypt, digest, isDigest, isEncrypted } from './machine';
import { encrypt as gcmEncrypt } from './gcm';

describe('machine digest', () => {
const key = 'a-high-entropy-machine-key-0123456789';

it('is deterministic and prefixed', () => {
const a = digest('value', key, 'ctx');
expect(a).toBe(digest('value', key, 'ctx'));
expect(isDigest(a)).toBe(true);
expect(a).not.toContain('value');
});

it('changes with the value, key and context', () => {
const base = digest('value', key, 'ctx');
expect(digest('other', key, 'ctx')).not.toBe(base);
expect(digest('value', 'another-key', 'ctx')).not.toBe(base);
expect(digest('value', key, 'other-ctx')).not.toBe(base);
});

it('keeps context and value unambiguous', () => {
expect(digest('bc', key, 'a')).not.toBe(digest('c', key, 'ab'));
});

it('does not treat hex hashes or random tokens as digests or ciphertext', () => {
expect(isDigest('abc123')).toBe(false);
expect(isEncrypted('abc123')).toBe(false);
});
});

describe('machine cipher', () => {
const key = 'a-high-entropy-machine-key-0123456789';

it('round-trips content', async () => {
const plaintext = 'https://hooks.example.com/webhook?x=1';
const enc = await encrypt(plaintext, key);
expect(enc.startsWith('m1:')).toBe(true);
expect(isEncrypted(enc)).toBe(true);
expect(await decrypt(enc, key)).toBe(plaintext);
});

Expand Down
22 changes: 22 additions & 0 deletions packages/core/src/encryption/machine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

import { gcm as aesGcm } from '@noble/ciphers/aes';
import { hkdf } from '@noble/hashes/hkdf';
import { hmac } from '@noble/hashes/hmac';
import { sha256 } from '@noble/hashes/sha2';
import { randomBytes, concatBytes, utf8ToBytes } from '@noble/hashes/utils';
import { Buffer } from '../buffer';
Expand All @@ -29,9 +30,30 @@ const KEY_LENGTH = 32;
const SCHEME_PREFIX = 'm1:';
const HKDF_INFO = utf8ToBytes('crypt.fyi/machine-metadata');

const DIGEST_PREFIX = 'hm1:';
// A separate HKDF info keeps digest keys independent of the encryption keys
// derived from the same machine key.
const DIGEST_HKDF_INFO = utf8ToBytes('crypt.fyi/machine-digest');

const deriveKey = (key: string, salt: Uint8Array): Uint8Array =>
hkdf(sha256, utf8ToBytes(key), salt, HKDF_INFO, KEY_LENGTH);

export const isEncrypted = (value: string): boolean => value.startsWith(SCHEME_PREFIX);

export const isDigest = (value: string): boolean => value.startsWith(DIGEST_PREFIX);

/**
* Deterministic keyed digest, for values that must be compared where the machine
* key isn't available (such as inside a Redis script). `context` scopes the
* digest to one use and must not contain a NUL byte, which separates it from
* `value` so the pair is encoded unambiguously.
*/
export const digest = (value: string, key: string, context: string): string => {
const digestKey = hkdf(sha256, utf8ToBytes(key), undefined, DIGEST_HKDF_INFO, KEY_LENGTH);
const message = concatBytes(utf8ToBytes(context), new Uint8Array([0]), utf8ToBytes(value));
return DIGEST_PREFIX + Buffer.from(hmac(sha256, digestKey, message)).toString('base64');
};

export const encrypt: Encrypt = async (content, key) => {
try {
const salt = randomBytes(SALT_LENGTH);
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/vault.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export type ProcessingMetadata = z.infer<typeof processingMetadataSchema>;

export const vaultValueSchema = z.object({
c: z.string().describe('encrypted content'),
h: z.string().describe('sha256 hash of the encryption key + optional password'),
h: z.string().describe('verification hash of the encryption key + optional password'),
m: processingMetadataSchema.optional(),
b: z.boolean().describe('burn after reading'),
dt: z.string().describe('delete token'),
Expand Down
125 changes: 124 additions & 1 deletion packages/server/src/app.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import type { AddressInfo } from 'node:net';
import { pino } from 'pino';
import { createRedisVault } from './vault/redis.js';
import { createTokenGenerator } from './vault/tokens.js';
import { createNopWebhookSender } from './webhook.js';
import { createNopWebhookSender, type Message } from './webhook.js';
import { Redis } from 'ioredis';
import { Client } from 'undici';
import { machine, type CreateVaultRequest } from '@crypt.fyi/core';
Expand Down Expand Up @@ -378,6 +378,129 @@ describe('app', () => {
expect(decryptedWhU).toBe('https://example.com/webhook');
});

describe('verification hash and delete token at rest', () => {
const m = { encryption: { algorithm: 'aes-256-gcm' as const } };
const createEntry = async (body: Partial<CreateVaultRequest> = {}) => {
const response = await testContext.client.request({
method: 'POST',
path: '/vault',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
c: 'secret-content',
b: false,
h: 'abc123',
ttl: 60_000,
m,
...body,
} satisfies CreateVaultRequest),
});
expect(response.statusCode).toBe(201);
return (await response.body.json()) as { id: string; dt: string };
};
const read = (id: string, h: string) =>
testContext.client.request({
method: 'GET',
path: `/vault/${id}?h=${encodeURIComponent(h)}`,
});
const remove = (id: string, dt: string) =>
testContext.client.request({
method: 'DELETE',
path: `/vault/${id}`,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ dt }),
});
const stored = async (id: string) =>
JSON.parse((await testContext.redis.get(`vault:${id}`))!) as Record<string, unknown>;

it('stores neither value in plaintext', async () => {
const { id, dt } = await createEntry();
const raw = (await testContext.redis.get(`vault:${id}`))!;

expect(raw).not.toContain('abc123');
expect(raw).not.toContain(dt);
expect(machine.isDigest((await stored(id)).h as string)).toBe(true);
expect(machine.isEncrypted((await stored(id)).dt as string)).toBe(true);
});

it('verifies with the original hash and rejects the stored digest', async () => {
const { id } = await createEntry();
const { h: storedH } = await stored(id);

expect((await read(id, storedH as string)).statusCode).toBe(400);
expect((await read(id, 'abc123')).statusCode).toBe(200);
});

it('does not let a stored delete token delete the entry', async () => {
const { id, dt } = await createEntry();
const { dt: storedDt } = await stored(id);

expect((await remove(id, storedDt as string)).statusCode).toBe(404);
expect((await remove(id, dt)).statusCode).toBe(200);
});

it('produces different digests for the same hash on different entries', async () => {
const [a, b] = await Promise.all([createEntry(), createEntry()]);
expect((await stored(a.id)).h).not.toBe((await stored(b.id)).h);
});

it('still serves entries written before this scheme', async () => {
const seed = async (id: string, dt: string) =>
testContext.redis.set(
`vault:${id}`,
JSON.stringify({ c: 'c', h: 'legacyhash', b: false, dt, ttl: 60_000, cd: Date.now(), m }),
'PX',
60_000,
);

await seed('legacy-read', 'legacy-dt-1');
expect((await read('legacy-read', 'wrong')).statusCode).toBe(400);
expect((await read('legacy-read', 'legacyhash')).statusCode).toBe(200);
await testContext.redis.del('vault:legacy-read');

await seed('legacy-delete', 'legacy-dt-2');
expect((await remove('legacy-delete', 'nope')).statusCode).toBe(404);
expect((await remove('legacy-delete', 'legacy-dt-2')).statusCode).toBe(200);
});

it('sends the original delete token to webhooks', async () => {
const messages: Message[] = [];
const vault = createRedisVault(
testContext.redis,
createTokenGenerator({ vaultEntryIdentifierLength: 20, vaultEntryDeleteTokenLength: 20 }),
{ send: async (message) => void messages.push(message) },
'foobar',
);
const { id, dt } = await vault.set({
c: 'c',
h: 'abc123',
b: false,
ttl: 60_000,
m,
wh: { u: 'https://example.com/webhook', r: true, fpk: false, fip: false, b: false },
});

await vault.get(id, 'abc123', '127.0.0.1');

expect(messages).toHaveLength(1);
expect(messages[0]?.dt).toBe(dt);
});

it('fails without using up attempts when the encryption key has changed', async () => {
const { id } = await createEntry({ fc: 1 });
const rotated = createRedisVault(
testContext.redis,
createTokenGenerator({ vaultEntryIdentifierLength: 20, vaultEntryDeleteTokenLength: 20 }),
createNopWebhookSender(),
'a-different-key',
);

await expect(rotated.get(id, 'abc123', '127.0.0.1')).rejects.toThrow();

expect((await stored(id)).fc).toBe(1);
expect((await read(id, 'abc123')).statusCode).toBe(200);
});
});

it('deletes vault entry', async () => {
const createResponse = await testContext.client.request({
method: 'POST',
Expand Down
Loading
Loading