Skip to content

feat(indexer): keyset pagination, holders, order book (api-server v2) - #4

Merged
erubboli merged 5 commits into
mintlayer:mainfrom
nullPointerEnjoyer:feat/indexer-v2-keyset-pagination
Oct 5, 2026
Merged

erubboli merged 5 commits into
mintlayer:mainfrom
nullPointerEnjoyer:feat/indexer-v2-keyset-pagination

Conversation

@nullPointerEnjoyer

Copy link
Copy Markdown
Collaborator

Implements the api-server v2 changes from mintlayer-core PR #2130 (commit 2684078c546, api-server 1.4.1), mirroring the existing SDK structure (one client method per endpoint, request-option pattern, typed responses, typed errors). No breaking changes to existing method signatures — the one behavioral fix is Transaction's pending-tx fields becoming Option so mempool entries decode.

New endpoints & response types

  • GET /statistics/coin/holders, GET /statistics/token/{id}/holders: address-balance holders listing (Holder { address, amount }, amounts rendered with the coin's/token's decimals). Unknown token → Error::TokenNotFound.
  • GET /order/pair/{pair}/book: aggregated order book per side (OrderBook/OrderBookLevel/OrderBookPrice, exact numer/denom price plus floored-toward-zero decimal). Side-specific cursors (ask cursor on a bid walk → Error::InvalidCursor). Truncation invariant encoded in the type: truncated: true always arrives with next_cursor: null, documented, and the pager stops on it. Coin tickers match case-insensitively; token ids are preserved exactly (bech32 is case-sensitive).
  • Statistics docs now state the four supply counters (circulating_supply, preminted, burned, staked) are always present; unwritten counters are zero.

Cursor pagination

  • Envelope type Page<T> { items, next_cursor } and a reusable Pager<T> walker (*_pager constructors for pools, the global transaction listing, both holders listings, and the order book): transparently follows next_cursor, stops on null, keeps its position across failed fetches (retriable), typed errors surface as Some(Err).
  • Page sizes clamped client-side to 1..=100 (MAX_NUM_ITEMS); server-side items=0/items>100 rejection mapped to Error::InvalidNumItems.
  • list_transactions_with_offset_mode (OffsetMode::Legacy/Absolute); combining cursor + offset_mode → Error::BadRequest (documented). Pools: cursors only with the default by_height sort (documented; other sorts keep the offset-based list_pools).
  • Cursor-overrides-offset semantics documented (not "fixed" client-side); page stability only guaranteed once the scanner is caught up (documented per endpoint).

Typed errors

Error::InvalidCursor, InvalidNumItems, BadRequest, TokenNotFound, mapped from exact {"error": ...} server bodies; anything else keeps the existing Error::Http fallback.

Tests & docs

  • 14 new wire tests (httpmock fixtures): full cursor walks (holders, pools, global transactions incl. pending null block_id), both book sides, truncated book stops the pager, last page, invalid-cursor/invalid-num-items/token-not-found, page-size clamping, retry-resume after a failed fetch, start_from rewind.
  • docs/indexer.md + README: per-endpoint cursor rules, error table with exact wire messages, runnable snippets; new examples/indexer-pagination.rs (holders, both-side book, transactions with cursor and offset_mode, pools — noting the sort/cursor restriction).

Validation: cargo test --all-features (107 tests) green, cargo clippy --all-targets --all-features and --no-default-features --all-targets 0 warnings, cargo fmt --check, cargo doc clean. Review loop (4-dimension adversarial reviewer fan-out) run recursively to 0 findings: cycle 1 → 11 findings, cycle 2 → 5, cycle 3 → 4, cycle 4/5 → 0.

Mirror the api-server v2 changes from mintlayer-core PR #2130
(api-server 1.4.1):

- statistics/coin and statistics/token/{id}: document that all four
  supply counters are always present (unwritten = zero)
- new holders listings: GET /statistics/coin/holders and
  GET /statistics/token/{id}/holders with cursor + offset support
- new order book: GET /order/pair/{pair}/book with side=ask|bid,
  aggregated price levels (exact rational + floored decimal string),
  side-specific cursors, and the truncated invariant
  (truncated => next_cursor null, walk cannot continue)
- shared cursor envelope Page<T> and a reusable Pager that follows
  next_cursor, stops on null, and clamps page sizes to 1..=100
- typed errors mapped from the server's error bodies: invalid cursor,
  invalid number of items, bad request, token not found
- Transaction.block_id/timestamp/confirmations are now Option<String>
  (the server emits JSON null for pending transactions)
- transactions: offset_mode=legacy|absolute for the offset listing;
  cursor and offset_mode are mutually exclusive (server 400)
- pools: cursor walk pinned to the default creation-height sort
- docs and a runnable example covering all four walks
- Pager: keep the resume cursor on a failed fetch (clone instead of
  take), so a retried walk resumes instead of restarting; pinned by a
  mid-walk 500 retry test.
- Widen pagination offsets to u64 to match the server contract
  (OffsetMode::Absolute offsets are global transaction indexes).
- CHANGELOG: document the v2 pagination API, typed errors, Transaction
  optionality break, and the pagination example.
- order_book_pager: document that a truncated book ends the walk
  indistinguishably from a complete one.
- docs: pools/transactions return plain arrays without a cursor param;
  quote the exact wire messages ("Invalid cursor", "Invalid number of
  items", "Bad request", "Token not found").
- Declare examples/indexer-pagination with required-features = ["indexer"]
  so --no-default-features --all-targets builds keep working.
- README: list the indexer-pagination example; docs: order_book_pager
  returns Result.
- Pager::next_page doc: buffered items survive once the walk is exhausted.
- Pager::start_from rewinds an exhausted pager (resets done/pending), so
  cursor() always matches what the next fetch uses; pinned by a test.
- OffsetMode::Absolute doc: state the precise upstream boundary (the page
  holds the transactions with global indexes below the offset).
- Coin-holder test fixtures: make atoms/decimal pairs self-consistent at
  the coin's 11 decimals.
- indexer-pagination example header: no longer claims an end-to-end walk.
@erubboli
erubboli self-requested a review October 5, 2026 10:38
@erubboli
erubboli merged commit 21b5c0f into mintlayer:main Oct 5, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants