Close /api/refresh, budget /api/contact, record git_sha, fix the Sources panel

[markrusch]

No methodology change. Nothing in index.py, normalise.py or weights.py is touched; reproduce gives 863 MATCH + 14 RETIRED on Python 3.11 and 3.13, and 1,775 of 1,775 published digests match.

What changes

/api/refresh (security, the important one). It dispatched daily.yml for any anonymous GET or POST. Because the 11:00 job skips a collector that already has an ok run that day (has_ok_run), a dispatch before 11:00 becomes the fixing. After 11:00 each call stores a new revision of all 21 series and commits the ~58 MB database. Now:

• POST only (a GET that starts a collection can be fired by a prefetcher or crawler)
• Authorization: Bearer <REFRESH_SECRET>, constant-time compare; closed while REFRESH_SECRET is unset
• refused before 11:00 UTC, whoever asks
• GitHub's error body no longer echoed

/api/contact. Site-wide budget of 10/hour, 40/day in the existing Upstash database (nothing about the sender is stored; fails open if Upstash is absent), length caps, plausible-email check before sending, line breaks stripped from the name before it becomes a subject.

Provenance. runs.git_sha has been null on all 791 runs. Collection and index runs now store HEAD, marked -dirty if src/ or config/ had uncommitted changes, falling back to GITHUB_SHA.

Sources panel. It listed the retired hand-kept rate cards as live and missed eight collectors in today's print. The list now follows collectors_for_daily, enforced by test_sources_panel.py. The live chip "8 of 5 providers" now reads "8 providers, 5 needed".

Before merging

1. In Vercel → compute-index → Environment Variables, add REFRESH_SECRET (long random string) if you want to keep the catch-up lever. Without it the endpoint refuses everything, which is a safe default since nothing on the site calls it.
2. Generated HTML is not in this PR; the next daily run regenerates index.html and latest.json.

Tests

• tests/test_api_functions.py runs both handlers under Node with fetch recorded (skipped where Node is absent; ubuntu-24.04 ships it). Six cases fail against the previous handlers.
• tests/test_provenance.py, tests/test_sources_panel.py.
• Full suite green on 3.11 and 3.13; ruff and mypy clean.

The CHANGELOG entry also records two found-not-fixed items: three fixings read before 11:00 (16 Aug, 12 Sep, 17 Sep), and a clock-dependent intraday smoke test that failed once locally.

🤖 Generated with Claude Code

https://claude.ai/code/session_013YAjeK25CLP46YirLiLjtg

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
compute-index	Ready	Preview	Oct 10, 2026 1:02pm UTC

[Copilot]

🔵 Needs a closer look

It changes authentication and rate-limiting logic on public endpoints, which is security-sensitive and warrants final human review despite being well-tested with no defects found.

0 open findings

What changed in this PR

This PR hardens two public Vercel serverless endpoints, adds code-commit provenance to pipeline runs, and corrects the dashboard's Sources panel. It does not touch the calculation path (index.py, normalise.py, weights.py), so no stored print changes and reproduce still matches every published digest.

Changes:

• /api/refresh: now POST-only, gated behind a constant-time Authorization: Bearer <REFRESH_SECRET> check (closed entirely when the secret is unset), refused before the 11:00 UTC fixing, and no longer echoes GitHub's error body.
• /api/contact: adds a site-wide Upstash-counted budget (10/hour, 40/day, fail-open), length caps, a plausible-email check, and strips line breaks from the name before it becomes a subject.
• Provenance & Sources panel: collection and index runs now record runs.git_sha (HEAD, -dirty when src//config/ are modified, falling back to GITHUB_SHA); the Sources panel is driven from collectors_for_daily() (enforced by a new test) and the live chip reads "N providers, M needed".

File	Description
site/​api/​refresh.js	Adds bearer-secret auth, before-fixing refusal, POST-only, stops leaking GitHub error body
site/​api/​contact.js	Adds Upstash-backed send budget, field length/email validation, name newline stripping
src/​tci/​db.py	New code_sha() helper (cached, never raises) resolving the running commit
src/​tci/​collectors/​base.py	Records git_sha on collector runs
src/​tci/​commands.py	Records git_sha on index runs
src/​tci/​outputs/​site.py	Reword live chip from "N of M providers" to "N providers, M needed"
config/​source_links.yaml	Rebuilds collector list to match collectors_for_daily(), retires static_yaml label
tests/​test_api_functions.py	Node-harness tests for both handlers
tests/​test_provenance.py	Tests code_sha() and run recording
tests/​test_sources_panel.py	Enforces panel ↔ collectors parity
CHANGELOG.md	Documents the above plus two found-not-fixed items

I verified the collector-name/order parity (including computable_collectors()), the Upstash pipeline response-index mapping, the constant-time comparison, the code_sha() exception/fallback paths, and that the new validation blocks email/subject header injection. I found no issues requiring changes.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.