Skip to content

feat(relay): deliver pubkey mentions to relay companions - #7793

Merged
jsibbison-square merged 1 commit into
mainfrom
jsib-260922-operator-listener-mentions-v2
Sep 28, 2026
Merged

jsibbison-square merged 1 commit into
mainfrom
jsib-260922-operator-listener-mentions-v2

Conversation

@jsibbison-square

@jsibbison-square jsibbison-square commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

We are introducing a companion service to the relay that needs pubkey event mentions for pubkeys it manages. This PR introduces an outbox for pubkey mentions that will be sent to these "operator listener" services.

Summary

Deliver mentions of registered pubkeys to deployment-global operator listeners.

  • Configure listeners with BUZZ_OPERATOR_LISTENERS; listeners manage target pubkeys through NIP-98-authenticated POST/DELETE /operator/listener/pubkeys requests.
  • During supported event ingestion, match p tags to registrations and insert one outbox row per listener in the event transaction.
  • A background worker claims up to 10 rows, sends NIP-98-signed notifications, and retries transient failures with exponential backoff for up to 9 attempts (idle polling backs off from 250 ms to 2 s).
  • Outbox rows are ineligible for delivery after 15 minutes and are reaped every 5 minutes. Registered pubkeys live for 30 days; re-registering renews them, and a daily reaper removes expired registrations.
  • Keep persistence app-managed: no database triggers, functions, foreign keys, or separate match queue.

Changed behavior

Listener outbox rows are enqueued inside the shared event-insert and thread-metadata transactions; enqueue failure rolls back the event. event_mentions behavior is unchanged: ordinary event/reaction mention indexing remains post-commit best effort with warning-only failures.

Testing

  • cargo fmt --all
  • cargo test -p buzz-db --lib
  • cargo test -p buzz-relay --lib config::tests::operator_listener_routes
  • cargo check -p buzz-db -p buzz-relay

@github-actions

github-actions Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is b0d6fb8ad27f6f255a5044e49ed0a59e11542914...1416c8713b8fdc957df791995dac49526bda0b78.
A new review must complete for this exact range. When manual authorization
is required, a Block organization member must comment exactly
@buzz-security-review 1416c8713b8fdc957df791995dac49526bda0b78 to authorize a new review.
Any previous review applies only to its recorded range.

@jsibbison-square
jsibbison-square force-pushed the jsib-260922-operator-listener-mentions-v2 branch 11 times, most recently from 4eacb6c to 43597fb Compare September 23, 2026 03:18
@jsibbison-square
jsibbison-square marked this pull request as ready for review September 23, 2026 04:53
@jsibbison-square
jsibbison-square requested a review from a team as a code owner September 23, 2026 04:53
@TheSentinel454

Copy link
Copy Markdown
Contributor

I reviewed this at exact head 43597fbfedb7820f4ad5f3da136071bbfcf2d20d. Two issues look worth addressing before merge:

P1: deterministic HTTP failures consume the full retry budget

In crates/buzz-relay/src/operator_listener.rs:281,299-326, every non-2xx response enters retry_or_fail. That means permanent 400/401/403/404 responses are retried up to MAX_DELIVERY_ATTEMPTS (9 total sends), despite the intended contract being to retry transient failures. We reproduced this through the live relay-to-companion path: a deterministic 400 generated nine POSTs before terminal removal.

Could we classify outcomes so transport errors, 408, 429, and 5xx retry, while ordinary deterministic 4xx responses fail immediately? Tests should distinguish a one-call terminal 400 from retrying 429/503 responses.

P1: default redirect following can silently complete delivery at the wrong endpoint

The reqwest client at crates/buzz-relay/src/operator_listener.rs:64-66 uses the default redirect policy, while the NIP-98 event is signed once for the configured URL. A redirect can change the request method or replay the body at another destination; if the final response is 2xx, deliver_one treats it as accepted and deletes the outbox row even though the configured signed endpoint never accepted that request. This creates a silent delivery-loss path and expands the destination boundary for mention metadata.

Could we construct the client with reqwest::redirect::Policy::none() and treat 3xx as a terminal endpoint/configuration failure? A regression test using the real reqwest transport should prove a redirected endpoint receives nothing and the delivery is not completed.

Architecture question (not classified as a defect)

Is deployment-global listener registration—and therefore release of mention metadata across community boundaries—an intentional operator-trust exception?

Migration 0049 explicitly says registrations span communities, while the isolated-community direction in VISION.md:52-56, VISION_REMOTE_AGENTS.md:15, and docs/multi-tenant-relay.md:9-21 points toward community-scoped boundaries. If the global scope is intentional, could that exception and its trust model be documented explicitly? If it is not intentional, would registrations and delivery be safer scoped to the originating community?

@jsibbison-square
jsibbison-square force-pushed the jsib-260922-operator-listener-mentions-v2 branch 2 times, most recently from 8bd65ce to 291c1cd Compare September 24, 2026 00:35
@jsibbison-square

Copy link
Copy Markdown
Contributor Author

@TheSentinel454 fixed "deterministic HTTP failures consume the full retry budget" and "default redirect following can silently complete delivery at the wrong endpoint".

Regarding "Architecture question" yes this is deliberately operator level as our service needs to be pushed information without knowing all the communities that the agents may be in. Therefore we cannot register with community fencing upfront. It's not a subscription registration in a normal user sense.

@jsibbison-square
jsibbison-square force-pushed the jsib-260922-operator-listener-mentions-v2 branch from 291c1cd to 501f93e Compare September 24, 2026 01:26
Comment thread crates/buzz-relay/src/operator_listener.rs
@jsibbison-square
jsibbison-square force-pushed the jsib-260922-operator-listener-mentions-v2 branch 2 times, most recently from c6c9ee0 to 398999b Compare September 24, 2026 23:45
Comment thread desktop/tests/e2e/virtualization.spec.ts Outdated
@jsibbison-square
jsibbison-square force-pushed the jsib-260922-operator-listener-mentions-v2 branch from 398999b to 1416c87 Compare September 28, 2026 00:04

@baxen baxen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approving. The core design holds up: mentions are queued in the same transaction as the event insert, only for new events, on both insert paths; leases and claim tokens fence out stale workers; and migration 0050 updates the fence-exclusion function before it creates the tables. I also ran the Postgres suites that the testing section skips, at 1416c871: buzz-db passed 306/306, including the 7 new operator_listener tests, migration parity and the tenant fence, and the buzz-relay operator routes passed 16/16.

None of the following blocks the merge. They're follow-ups, in a later commit or a separate PR, whichever you prefer:

1. Open the transaction only for mention kinds (store/event.rs:352)
insert_event now opens a transaction for every write. That adds BEGIN/COMMIT round trips to side-effect, git, audio and workflow events, and none of them can ever queue a mention. Check is_listener_mention_kind first and keep the plain path for everything else.

2. Claim only rows this pod can route (claim_deliveries, store/operator_listener.rs:202)
Pass the configured listener pubkeys into the claim and add AND listener_pubkey = ANY($n). That lets you delete release_unroutable_delivery, the release path in the worker and their tests. It also fixes a real problem: when a listener is removed from config, its registrations keep producing rows for up to 30 days, and every 30s those rows are claimed, released and logged as warnings until they age out.

3. Fail fast on a bad BUZZ_OPERATOR_LISTENERS (config.rs:857)
A malformed value currently logs an error and starts the relay with the feature disabled. Registrations return 403 and delivery silently stops, and the only sign is a single log line at startup. Other operator-identity config returns ConfigError and stops startup, including RELAY_OWNER_PUBKEY, RELAY_OPERATOR_PUBKEYS and BUZZ_PUSH_GATEWAY_DELIVERY_URL, for the reason given in the comment on RELAY_OPERATOR_PUBKEYS. Making this ? would match them. It also lets you revert the log-capture test helper refactor, since the tests can simply assert is_err().

4. Drop the DeliveryStore / DeliveryTransport traits (operator_listener.rs:97-120)
Each trait has one production implementation, so the indirection only exists for tests, along with the mocks and the hand-rolled HTTP request parser. push_runtime.rs tests the same kind of signed outbound POST against a real local axum server. Doing the same here removes about 200 lines and tests the real reqwest and NIP-98 signing path, not a mock of it.

@jsibbison-square
jsibbison-square merged commit c4c8600 into main Sep 28, 2026
81 checks passed
@jsibbison-square
jsibbison-square deleted the jsib-260922-operator-listener-mentions-v2 branch September 28, 2026 03:04
jsibbison-square added a commit that referenced this pull request Sep 28, 2026
## Summary

- Keep non-listener-mention event writes on the plain insert path
without opening a transaction.
- Preserve the transaction around listener mention event insertion and
outbox enqueue.

## Validation

- `cargo fmt --all -- --check` (Hermit)
- Tests not run.

Follow-up to Baxen’s review of #7793, point 1.

Signed-off-by: Implementor <691cca7a870db1dad6990d5938d1b4a2a2ca9647ca81290b536c17b06b2e473f@buzz.block.builderlab.xyz>
Co-authored-by: Implementor <691cca7a870db1dad6990d5938d1b4a2a2ca9647ca81290b536c17b06b2e473f@buzz.block.builderlab.xyz>
jsibbison-square added a commit that referenced this pull request Sep 28, 2026
Addresses point 3 of [previous
comment](#7793 (review))
"3. Fail fast on a bad BUZZ_OPERATOR_LISTENERS (config.rs:857)"

## Summary
- Propagate malformed `BUZZ_OPERATOR_LISTENERS` values from
`Config::from_env` as `ConfigError`, including invalid UTF-8.
- Update startup config tests to assert returned errors and restore the
original admin warning capture helper, removing the shared helper
introduced only for listener log assertions.
- Cover valid and absent listener values through `Config::from_env`.

## Context
Follow-up to the review of #7793 specifically [reverting some config
logging
parts](https://github.com/block/buzz/pull/7793/changes#diff-eaa0eeb209ff1ad6212ffa56d3a983bb732858647c9fb227ef7483a3363b5a71).
Invalid listener configuration currently logs an error and silently
disables listener delivery.

## Verification
- `cargo fmt --all -- --check` passed with Hermit.

Signed-off-by: Implementor <691cca7a870db1dad6990d5938d1b4a2a2ca9647ca81290b536c17b06b2e473f@buzz.block.builderlab.xyz>
Co-authored-by: Implementor <691cca7a870db1dad6990d5938d1b4a2a2ca9647ca81290b536c17b06b2e473f@buzz.block.builderlab.xyz>
wpfleger96 pushed a commit that referenced this pull request Sep 28, 2026
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>

* origin/main:
  fix(relay): fail startup on invalid operator listener config (#7933)
  fix(db): limit event transactions to listener mention kinds (#7932)
  feat(relay): deliver pubkey mentions to relay companions (#7793)
  docs(protocol): propose simplified channel artifacts (#7791)
  feat(push): support configurable HTTP(S) delivery URLs (#7877)
  fix(ci): select runtime suites from PR changes only (#7843)
  test(desktop): synchronize upload edit smoke test (#7903)

Signed-off-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
wpfleger96 pushed a commit that referenced this pull request Sep 28, 2026
* origin/main:
  🤖 docs(nip-fi): remove implementation references from the spec (#7912)
  fix(relay): fail startup on invalid operator listener config (#7933)
  fix(db): limit event transactions to listener mention kinds (#7932)
  feat(relay): deliver pubkey mentions to relay companions (#7793)
  docs(protocol): propose simplified channel artifacts (#7791)
  feat(push): support configurable HTTP(S) delivery URLs (#7877)
  fix(ci): select runtime suites from PR changes only (#7843)
  test(desktop): synchronize upload edit smoke test (#7903)

Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants