Skip to content

perf(identity): index pending outbound counts by agent - #1075

Open
RauPro wants to merge 1 commit into
tokencanopy:mainfrom
RauPro:perf/129-pending-count-index
Open

RauPro wants to merge 1 commit into
tokencanopy:mainfrom
RauPro:perf/129-pending-count-index

Conversation

@RauPro

@RauPro RauPro commented Oct 8, 2026

Copy link
Copy Markdown

Summary

Addresses item 1 of #129: add a partial messages(agent_id) index for status = 'pending_review' AND direction = 'outbound', matching ListAgentsByUser's existing all-time pending_count subquery. DNS error visibility is handled separately in #1032; this PR does not close the whole issue.

The expiry index leads with approval_expires_at, so it cannot directly seek to one agent's holds. The new index supports that lookup without adding a cached counter or changing the query's results. It intentionally adds neither an age cutoff nor a deleted_at predicate absent from the query.

Measurement

Measured the actual SQL emitted by ListAgentsByUser on PostgreSQL 16.15, using EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) and default planner settings. Synthetic fixture: 100 agents across 5 owners, 160,000 historical sent messages, 2,000 outbound holds and 2,000 inbound holds; the selected owner has 20 agents. Rows are 14 days old and carry a 512-byte synthetic raw body.

Both sides used the same rows after VACUUM (ANALYZE); timings are medians of five warm executions after one warm-up.

Measurement Before After
Complete list-agents SQL 1.357 ms 0.201 ms
Complete query shared buffer hits 517 286
Pending-count subplan shared buffer hits 272 41

The pending subplan changed from a bitmap intersection of the agent/history and global review indexes to an index-only scan on idx_messages_agent_pending, with zero heap fetches in this vacuumed fixture. The new index occupied 40 KB. These are local synthetic results, not a production latency guarantee.

Operational risk

Migration 132 uses CREATE INDEX CONCURRENTLY and the existing e2a:no-transaction runner, avoiding a write-blocking index build. Building still consumes I/O and can wait for existing transactions; matching holds incur index-maintenance cost. The migration documents invalid-index recovery, and the runner already refuses to record an invalid build as applied.

No public API or SDK changes.

Test plan

New DB-backed tests verify the embedded migration and idempotent reapplication, index validity/readiness and definition, and pending counts across agents, owners, directions, terminal states, old holds and a transition to sent. Timing and planner-choice assertions are deliberately kept out of CI.

Verified locally with Go 1.26.9 and PostgreSQL 16.15:

  • New regression tests and existing non-transactional migration/invalid-index recovery tests
  • go test -tags integration ./internal/identity -count=1 -timeout=11m
  • go test ./...
  • make fmt-check
  • scripts/check-repository-text-integrity.sh
  • scripts/check-no-committed-credentials.sh
  • git diff --check

Add a partial agent_id index matching ListAgentsByUser's all-time
outbound pending_review count. Build it concurrently through the
existing non-transactional migration runner.

Verify index validity and idempotent reapplication, plus per-agent and
per-owner counts across directions, statuses and resolved holds.

Refs tokencanopy#129 (pending-count index; DNS handling is covered by tokencanopy#1032).

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Raul Ernesto Guillen Hernandez <37725305+RauPro@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant