Skip to content

Add parallel mod_to_acct_transfers_summarized table#79

Merged
Alann27 merged 1 commit into
mainfrom
feature/summarize-mod-to-acct-transfers
Mar 25, 2026
Merged

Add parallel mod_to_acct_transfers_summarized table#79
Alann27 merged 1 commit into
mainfrom
feature/summarize-mod-to-acct-transfers

Conversation

@jorgecuesta

Copy link
Copy Markdown
Collaborator

Summary

  • Creates mod_to_acct_transfers_summarized table that groups by (block_id, recipient_id, op_reason, denom, service_id), reducing row count ~45x on mainnet (1.6M → 36K per settlement block)
  • Writes in parallel alongside the existing mod_to_acct_transfers table — no existing queries or functions are modified
  • Adds summarizeTransfers pure function that aggregates amount (BigInt sum) and transfer_count per group key

Files changed

  • src/mappings/dbFunctions/modToAcctTransfers.ts — new table DDL with 3 indexes
  • src/mappings/dbFunctions/index.ts — register table creation at startup
  • src/mappings/pocket/relays.ts — summarization logic + parallel bulk insert

Validated in production

  • Beta: 5+ settlement blocks, all sum(transfer_count) match original row counts, all sum(amount) per recipient match
  • Mainnet: 1 settlement block (680793), 1,656,445 original rows → 36,252 summarized rows (45.7x), perfect consistency

What is NOT changed

  • mod_to_acct_transfers table, functions, queries — all untouched
  • _handleEventClaimSettled — untouched
  • bulkInsertModToAcctTransfers — untouched

Next steps (future PRs)

  • Backfill historical data
  • Validate completeness
  • Switch functions to use summarized table
  • Drop old table

Create a new summarized table that groups mod_to_acct_transfers by
(block_id, recipient_id, op_reason, denom, service_id), reducing row
count ~45x on mainnet. The existing table and all queries are untouched;
this writes in parallel for future cutover.

- Add createModToAcctTransfersSummarizedTableFn with 3 indexes
- Register table creation at startup in createDbFunctions
- Add summarizeTransfers pure function and bulkInsertSummarizedTransfers
- Wire into handleEventClaimSettled Promise.all
@jorgecuesta jorgecuesta requested a review from Alann27 March 25, 2026 21:03
@jorgecuesta jorgecuesta self-assigned this Mar 25, 2026
@jorgecuesta jorgecuesta added the enhancement New feature or request label Mar 25, 2026
@Alann27 Alann27 merged commit cd39787 into main Mar 25, 2026
2 of 3 checks passed
@Alann27 Alann27 deleted the feature/summarize-mod-to-acct-transfers branch March 25, 2026 21:05
jorgecuesta added a commit that referenced this pull request Jun 9, 2026
## Summary

Indexes the consensus-breaking event-payload additions shipped in
poktroll **v0.1.34**
([pokt-network/poktroll#1952](pokt-network/poktroll#1952)).
All additive proto fields + one new event; all changes are
backward-compatible (handlers fall back to prior derivation when a field
is absent).

## Changes

### 1. New `EventValidatorRewardDistribution` entity + handler
Emitted once per bonded validator per `op_reason` per settlement block
(~40/block, bounded by validator-set size — preserves the #1758
event-count reduction). Carries the canonical per-validator commission /
delegator split.

- `schema.graphql`: new `@entity` (session end height, op_reason,
validator operator/account addrs, commission rate, pool-share /
commission / self-delegation / delegators / total-delegated-stake upokt
amounts, num delegators).
- `relays.ts`: `handleEventValidatorRewardDistribution` parses all 11
attributes (`*_upokt` = bare integer strings, `commission_rate` =
decimal string, `op_reason` = enum name, `session_end_block_height` =
int64-as-string).
- Registered in `handlers.ts` `EventHandlers`; dispatched in
`indexer.manager.ts` `indexRelays`.

**Why it matters — cross-delegation accounting:** we already bucket
validator/delegator rewards from `EventSettlementBatch`
(`VALIDATOR_REWARD_OP_REASONS`). The bank-batch accumulator keys on
`(recipient, op_reason)`, so when the same `pokt` address is **both** a
validator account **and** a delegator on another validator, its income
is bucketed under the validator-side `op_reason` — over-counting
VALIDATOR, under-counting DELEGATOR. This event gives the unambiguous
per-validator split. The entity doc-comment in `schema.graphql` records
the caveat.

### 2. Prefer chain `num_estimated_relays` (field 13)
`EventClaimCreated` (`_handleMsgCreateClaim`) and `EventClaimExpired`
(`_handleEventClaimExpired`) computed `num_estimated_relays` via raw
`CUPR = floor(claimed / numRelays)` — **divide-by-zero when `num_relays
== 0`**. Both now route through the existing
`_computeNumEstimatedRelays` (prefers the chain-emitted value, falls
back to deriving from compute units for pre-upgrade blocks).
`EventClaimSettled` already did this.

### 3. Prefer chain `supplier_stake_after_slash` (field 9)
`EventSupplierSlashed.afterStakeAmount` now prefers the chain-emitted
post-slash stake; falls back to `trackedStake − penalty` for pre-upgrade
events (also robust when tracked stake drifts on pruned nodes).

## Notes

- Generated models / proto-interfaces are gitignored (regenerated at
build). `schema.graphql` is the tracked source of truth — `subql
codegen` confirmed clean (`EventValidatorRewardDistribution generated
!`).
- Edited line ranges + the new handler type-check clean. (Full local
`tsc`/lint floods on missing generated proto-interfaces, which are
produced by the buf + subql codegen pipeline in CI.)

## Follow-up (not in this PR)
Rewire the `mod_to_acct_transfers_summarized` rollup (#79) to source
VALIDATOR-vs-DELEGATOR totals from `EventValidatorRewardDistribution`
instead of `EventSettlementBatch`. The data is now indexed; the rollup
change is left as a separate scoped change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Jorge Cuesta <jorge.s.cuesta@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants