Skip to content

Extend targeted arXiv retrieval experiment to two bounded pages - #124

Merged
basicmi merged 2 commits into
masterfrom
arxiv-targeted-experiment-deeper-pages
Nov 17, 2026
Merged

basicmi merged 2 commits into
masterfrom
arxiv-targeted-experiment-deeper-pages

Conversation

@basicmi

@basicmi basicmi commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Motivation

Run #14 showed all four production arXiv shards reaching their 100-record cap. The isolated targeted-query experiment also saturated its one-page (50-record) cap in all four groups. Its four extra API requests found 3 additional known-item seeds and increased the curated observed-window hit rate from 56.25% to 75%, but did not measure precision.

Changes

  • Add a per-query-group max_pages setting to the isolated experiment, bounded to 1–2 pages.
  • Configure the four experimental query groups for two pages of up to 50 raw entries each.
  • Preserve the existing production source registry and one-run/45-day scope.
  • Preserve pagination diagnostics and full candidate results for deterministic human-review sampling.
  • Add offline tests that verify second-page offsets, request counts, deduplication, incremental candidates and page-cap reporting.
  • Add a short experiment protocol document.

Bounded cost

At most 8 arXiv API requests and 400 raw entries per run (4 groups × 2 pages × 50 results), with the existing three-second spacing between pages and query groups. The experiment remains isolated from production discovery and review queues.

Success criteria

  • Quantify incremental unique papers and known-seed hits beyond page one.
  • Identify groups that still saturate page two.
  • Produce a larger, reproducible review population without claiming precision until independent labels are supplied.

Guardrails

No production arXiv queries, hardware-support records, benchmark results or formal registries are changed.

@basicmi
basicmi merged commit 79d99be into master Nov 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant