As of 2026-07-17. Single-page operational state for any agent (or human) picking up work in this repo. Read this first. Then read whichever pointer below matches your task. Anything in reports/methodology_history.md past §0 is history — useful when investigating why, not what.
Maintenance rule. Update this file when any of {deployed methodology, served τ range, active workstream, headline metrics, deployment artefact path, wire format} changes. Changes that don't move any of those don't belong here. Long rationale belongs in
reports/methodology_history.md; this file links to that, not the other way around.
Deployed methodology: M6 — Locally-Weighted Conformal (LWC) + per-symbol σ̂ EWMA HL=8 + Mondrian-by-regime_pub + δ-shifted c(τ) bump. Python serving path is live; Rust parity is complete (180/180 Python↔Rust cases across both M5 and M6 paths). The Anchor on-chain program serves the M5 reference path on devnet; M6 on-chain enablement is pending the next publisher release.
Headline at τ=0.95 (OOS 2023+, 1,730 rows × 173 weekends × 10 tickers): realised 0.950, half-width 370.6 bps, Kupiec p=0.956, Christoffersen p=0.603, per-symbol Kupiec 10/10, held-out LOSO coverage 0.9497 ± 0.0128 (cross-symbol SD). Coverage holds Kupiec at every served τ ∈ {0.68, 0.85, 0.95, 0.99}.
Served τ range: [0.68, 0.99]. Default deployment τ = 0.85. Paper 1 headline τ = 0.95.
Wire format: PriceUpdate Borsh layout preserved across v1 → M5 → M6. forecaster_code = 2 = M5 Mondrian (live on-chain, devnet). forecaster_code = 3 = M6 LWC (live in the Rust serving stack at 180/180 parity; on-chain enablement gated on the next publisher release).
Deployment artefacts (under data/processed/):
| File | Role |
|---|---|
lwc_artefact_v1.{parquet,json} |
M6 canonical — what's served today |
lwc_artefact_v1_frozen_20260504.{parquet,json} |
SHA-256-stamped freeze used by the forward-tape harness (sha 7b86d17a76912aa0…) |
mondrian_artefact_v2.{parquet,json} |
M5 reference baseline (Paper 1 §7 ablation comparator) |
forward_tape_v1.parquet |
Accumulated forward weekends past the freeze cutoff |
lwc_artefact_v1_archive_baseline_k26_*.{parquet,json} |
Archival K=26 σ̂ variant (pre-EWMA promotion) |
20 deployment scalars total: 12 trained per-regime quantiles + 4 OOS-fit c(τ) bumps + 4 walk-forward-fit δ(τ) shifts. All zero δ under M6 — per-symbol scale standardisation closed the cross-split calibration gap that made δ load-bearing under M5.
| Workstream | Driving doc | Status |
|---|---|---|
| Paper 1 (coverage-inversion) → arXiv | research/coverage-inversion/ |
v2 structural rewrite complete: 4-reviewer adversarial pass addressed (incl. the matched-history tokenized-tracking reframe), figures redesigned, 63-page PDF builds via build/build.py --v2. Near human-review / arXiv submission; seeking a q-fin.RM endorsement (arxiv-endorsement/, gitignored). |
| M6 Rust parity | reports/active/m6_refactor.md |
✅ Complete — 180/180 Python↔Rust. On-chain M6 enablement (forecaster_code = 3) gated on the next publisher release. |
| Paper 3 — liquidation policy | research/liquidation-policy/ |
In flight. Three-claim structure (Geometric / Structural / Empirical). |
| Paper 4 — forward data capture | research/oracle-conditioned-amm/scryer_pipeline_plan.md |
Owned by scryer (item 51). Soothsayer consumers wait until parquet rows land. |
| Devnet publish path | crates/soothsayer-publisher, programs/ |
Router v0 deployed devnet 2026-04-29 at AZE8HixpkLpqmuuZbCku5NbjWqoQLWhPRTHp8aMY9xNU. Publish→read-back not yet wired end-to-end (initialize runner is a TODO; see docs/devnet-quickstart.md). |
| Forward-tape harness | scripts/run_forward_tape_harness.sh |
Live on launchd, fires weekly Tuesday. N=11 forward weekends (2026-05-01 → 2026-07-10): pooled Kupiec passes all four anchors, per-symbol 10/10 at τ=0.95. |
| AMM design-partner onboarding | docs/INTEGRATION.md |
Stubs scaffolded (integration guide + devnet quickstart + README section); content gated on the Paper 1 release. ROADMAP Phase 2. |
| Repo public-share prep | this file | ✅ Papers moved to research/; committed secrets removed; third-party papers + internal drafts untracked. |
Backlog of candidate workstreams sits in reports/active/validation_backlog.md. Treat that as scratch — anything that sticks gets folded into reports/methodology_history.md.
| Task | First file | Then |
|---|---|---|
| Understand current deployed methodology | reports/methodology_history.md §0 |
latest dated entry under §1 |
| Paper 1 revision | research/coverage-inversion/{section}.md |
reports/m6_validation.md for the evidence pack |
| Paper 3 (liquidation policy) | research/liquidation-policy/plan.md |
docs/protocol_semantics_kamino_xstocks.md for the verified Kamino semantics |
| Paper 4 (oracle-conditioned AMM) | research/oracle-conditioned-amm/plan.md |
research/oracle-conditioned-amm/scryer_pipeline_plan.md |
| M6 Phase 7 (Rust port) | reports/active/m6_refactor.md §7 |
crates/soothsayer-oracle/ (M5 path is the parity reference) |
| Forward-tape monitoring | scripts/run_forward_tape_harness.sh |
launchd/com.adamnoonan.soothsayer.forward-tape.plist + reports/m6_forward_tape_*.md |
| Adding a new robustness test on M6 | reports/active/phase_7_results.md (template) |
src/soothsayer/backtest/calibration.py dispatcher |
| Adding a new data source | ../scryer wishlist (do not fetch from Soothsayer) |
docs/sources/_template.md, then docs/scryer_consumer_guide.md |
| Devnet / on-chain publish | crates/soothsayer-publisher/ |
programs/soothsayer-oracle-program/, programs/soothsayer-router-program/ |
| Kamino integration demo | crates/soothsayer-demo-kamino/ |
reports/demo_kamino_comparison.md |
| Reading parquet from scryer | docs/scryer_consumer_guide.md |
src/soothsayer/sources/scryer.py helpers |
| Building / regenerating the M6 artefact | scripts/build_lwc_artefact.py |
sidecar JSON schema documented in reports/methodology_history.md 2026-05-04 entry |
data/processed/lwc_artefact_v1.{parquet,json}is what is served. Don't break or rename without coordinating.- The wire-format invariance guarantee (
PriceUpdateBorsh layout) is the consumer contract. Any change to it is a breaking on-chain change and needs a migration plan. - σ̂ rule = EWMA HL=8 as of 2026-05-04 (promoted from K=26 trailing window). Column name
sigma_hat_sym_pre_friis preserved across the swap, so consumers don't need to know which σ̂ rule is live. - Forward-tape harness on launchd validates the frozen freeze on each new closed weekend. If you change the freeze, update
scripts/freeze_lwc_artefact.pyand let the auto-discovery glob pick it up. - Hard rule: all upstream data fetching goes through scryer. See
CLAUDE.mdrule #1.
- M5 (Mondrian-only, no per-symbol scale standardisation) is the named reference baseline for the §7 ablation. Code path stays alive (
Oracle.fair_value,forecaster_code = 2). It is not what is served today. - v1 hybrid Oracle (F1_emp_regime + per-target buffer schedule) was retired 2026-05-XX. Diagnostic scripts moved to
scripts/v1_archive/. - 24×
reports/v1b_*.mdare frozen evidence snapshots tied to the M5 baseline. Read only when chasing a specific historical claim. - Dual-profile (M6a + M6b2) architecture described in earlier revisions of
reports/active/m6_refactor.mdis paused. The current contents of that file are the LWC promotion plan that supersedes it. - AMM-track shipping (M6a) is deferred indefinitely. Re-opens on either (a) a Sunday-Globex republish architecture (W8b, scryer-fetcher-gated) or (b) V3.1 F_tok tape accumulating ≥150 weekends (W8c, ETA Q3–Q4 2026).
- Pre-M5 reports (
v1_chainlink_bias.md,v3_bakeoff.md,v11_cadence_verification.md,phase1_week{1,2}.md) are historical. Seereports/INDEX.mdfor the current/historical classification.
- No upstream data fetching in Soothsayer — read scryer parquet from
SCRYER_DATASET_ROOT. - New data sources land in scryer first.
- Analysis reads parquet, not raw API output.
- Preserve
_schema_version,_fetched_at,_source,_dedup_key. - Soothsayer-derived datasets use experiment-versioned venues (
soothsayer_v{N}). - Don't restore deleted ingest code — migrate callers to scryer parquet.