Purpose: Track spec requirements to implementation status, PRs, and evidence. Canonical spec: master_architecture_spec.md.
- Spec requirement: Short label from the master spec or component file/section.
- Component file/section: Where the requirement is defined (for traceability).
- Implementation status: Not started | In progress | Done | Blocked.
- PR link: URL or PR number once implemented.
- Files changed: Key modules/files touched.
- Tests: Unit / integration / statistical tests added or updated.
- Evidence/hashes: Artifact hashes, dataset_id, or run IDs that prove the behavior.
- Notes: Caveats, follow-ups, or dependencies.
Update rows as work completes. Keep the master spec unchanged; extend component specs only when the spec itself evolves.
| Spec requirement | Component file/section | Implementation status | PR link | Files changed | Tests | Evidence/hashes | Notes |
|---|---|---|---|---|---|---|---|
| Research mechanism specs (Report A & B: goal, inputs, outputs, assumptions, validation, failure modes) | research_mechanisms.md | Not started | |||||
| Pipeline contract: Ingestion → Reporting (inputs, outputs, invariants, error handling per stage) | pipeline_contracts.md | Not started | |||||
| Baseline + refined dependency graph (Mermaid) | dependency_graph.md | Not started | |||||
| Research ↔ repo mapping table (mechanism → stage, status, changes, interfaces, tests, acceptance) | research_repo_mapping.md | Not started | |||||
| Schema evolution: schema_migrations, factor_model_runs, factor_betas, residual_returns, regime_* | schema_plan.md | In progress | migrations_v2.py (factor_); migrations_phase3.py (regime_) | test_migrations_*.py | factor_* in v2; regime_* in phase3 only | ||
| Migration strategy: versioned runner, backward compatibility, rollback | schema_plan.md | Not started | |||||
| Interface: Residualizer (as_of_lag_bars, leakage hardening) | interfaces.md | Not started | |||||
| Interface: RegimeDetector (fit/predict, filter-only in test) | interfaces.md | Done | crypto_analyzer/regimes/regime_detector.py | test_regime_detector.py | Phase 3 Regimes slice | filter-only; smooth raises in validation | |
| Interface: ExecutionCostModel (unified spread/impact) | interfaces.md | Not started | |||||
| Interface: MultipleTestingAdjuster (BH/BY) | interfaces.md | Not started | |||||
| Interface: Bootstrapper (block + stationary, seed) | interfaces.md | Not started | |||||
| Unit tests: Residualizer, RegimeDetector, ExecutionCostModel, MultipleTestingAdjuster, Bootstrapper | testing_acceptance.md | Not started | |||||
| Integration tests: ingestion→report, deterministic rerun, walk-forward leakage | testing_acceptance.md | Not started | |||||
| Statistical tests: null suite, permutation/placebo, multiple-testing correction, acceptance thresholds | testing_acceptance.md | Not started | |||||
| Versioning: SemVer rules, config_version, model artifact versioning, repro metadata in DB | versioning_release.md | Not started | |||||
| Performance: hotspots, runtime expectations, caching plan, SQLite limits + migration path | performance_scale.md | Not started | |||||
| Risk audit: leakage vectors, overfitting, regime dependence, capacity illusions, what NOT to implement | risk_audit.md | Not started | |||||
| Phase 1: Patch leakage (causal residualizer, as_of_lag_bars; quarantine lookahead) | phased_execution.md, master_architecture_spec.md | Done | (PR link) | crypto_analyzer/factors.py, crypto_analyzer/alpha_research.py | tests/test_leakage_sentinel.py | Leakage sentinel: causal path no abnormal IC; allow_lookahead=False default | |
| Phase 1: ValidationBundle contract + reportv2 emit per signal | phased_execution.md | Done | (PR link) | crypto_analyzer/validation_bundle.py, cli/research_report_v2.py | run.ps1 verify | Per-signal bundle JSON + IC/decay/turnover CSVs (relative paths) | |
| Phase 1: Deterministic rerun integration test | phased_execution.md | Done | (PR link) | tests/test_reportv2_deterministic_rerun.py, crypto_analyzer/timeutils.py, crypto_analyzer/artifacts.py, crypto_analyzer/governance.py | run.ps1 verify | test_deterministic_rerun_identical_bundle_and_manifest; env CRYPTO_ANALYZER_DETERMINISTIC_TIME | |
| Phase 1: Unify cost model (ExecutionCostModel) | phased_execution.md | Done | (PR link) | crypto_analyzer/execution_cost.py, crypto_analyzer/portfolio.py, cli/backtest.py | tests/test_execution_cost.py, run.ps1 verify | Same inputs → identical net_returns; higher turnover → higher costs; missing liquidity fallback | |
| Phase 2: schema_migrations + versioned migration runner (backup/restore) | schema_plan.md, phased_execution.md | Done | (PR link) | crypto_analyzer/db/migrations_v2.py, crypto_analyzer/db/migrations.py, ingest/init.py, read_api.py | test_migrations_v2_on_empty_db_creates_tables_and_records, test_migrations_v2_rerun_idempotent, test_migrations_v2_in_memory_no_backup, test_migrations_v2_failure_restores_backup | run_migrations(conn, db_path) from ingest/read_api; backup = copy on-disk DB, restore on failure; failure test asserts schema_migrations + factor_model_runs row count unchanged | No backup when db_path=None (log); idempotent CREATE INDEX IF NOT EXISTS |
| Phase 2: factor_model_runs, factor_betas, residual_returns + materialize path | schema_plan.md | Done | (PR link) | crypto_analyzer/factor_materialize.py, crypto_analyzer/factors.py (causal_rolling_ols) | test_factor_run_id_deterministic, test_materialize_factor_run_writes_tables, test_materialize_deterministic_under_fixed_time, test_materialize_idempotent_same_run_id, test_as_of_lag_bars_must_be_at_least_one, test_causal_residuals_no_future_data | factor_run_id = hash(canonical JSON + dataset_id); CRYPTO_ANALYZER_DETERMINISTIC_TIME → same row counts and ordered content; as_of_lag_bars>=1 enforced in causal_rolling_ols | Determinism: env CRYPTO_ANALYZER_DETERMINISTIC_TIME for reproducible materialize |
| Phase 2: MultipleTestingAdjuster (BH/BY) + registry (adjusted p-values, family) | interfaces.md, research_repo_mapping.md | Done | (PR link) | crypto_analyzer/multiple_testing_adjuster.py, cli/research_report_v2.py | test_bh_golden, test_by_more_conservative_than_bh, test_adjust_monotonicity, test_adjust_reproducible, test_no_family_empty_unchanged, test_no_family_nan_handling | reportv2: p_value_raw_ |
Explicit family_id = hash(config+signals+horizons+grid) recommended follow-up |
| Phase 2: Stationary bootstrap option + method/seed in artifacts | interfaces.md | Done | (PR link) | crypto_analyzer/statistics.py | tests/test_statistics_research.py | method=stationary, seed → reproducible; significance_summary returns bootstrap_method, bootstrap_seed, block_length | |
| Phase 2: Null suite runner (random ranks, permuted signal, block shuffle) | testing_acceptance.md | Done | (PR link) | crypto_analyzer/null_suite.py, cli/null_suite.py, scripts/run.ps1 | test_null_suite_produces_artifacts, test_null_1_deterministic, test_null_2_permutes_per_row, test_null_3_block_shuffle_reorders_rows | CLI: .\scripts\run.ps1 null_suite; artifacts: null_ic_dist.csv, null_sharpe_dist.csv, null_pvalues.json; fixed seed → deterministic output | Nulls built without future returns; block shuffle preserves within-block dependence |
| Phase 2: Spread proxy + participation impact + capacity curve | phased_execution.md | Done | (PR link) | crypto_analyzer/execution_cost.py | test_spread_increases_with_vol_and_lower_liquidity, test_impact_increases_with_participation, test_capacity_curve_multipliers, test_same_inputs_identical_net_returns, test_missing_liquidity_conservative_fallback | spread_vol_scale=0, use_participation_impact=False default (opt-in); capacity_curve fixed fee/slippage, varies notional multiplier only | Monotonicity: lower liq/higher vol → higher spread; higher participation → higher impact |
| Phase 2: E2E integration (migrations → materialize → registry → null suite) | phased_execution.md | Done | (PR link) | tests/test_phase2_integration.py | test_phase2_e2e_migrations_factor_registry_null_artifacts | Temp DB → run_migrations(conn, db_path) → materialize_factor_run → record_experiment_run → run_null_suite → write_null_suite_artifacts | Contract frozen in phase2_pr_description.md |
| Phase 3 checklist: regime models, Kalman beta, RC, promotion workflow (Slices 1–5 done); remaining: sweep registry hardening, execution realism gates, perf (measured) | phased_execution.md | In progress | Slices 1–5 done; sweep registry hardening done (PR 1); remaining: capacity-aware acceptance, factor/regime cache + profile/accelerate | ||||
| Phase 3 Sweep registry hardening (PR 1): sweep_families + sweep_hypotheses tables; hypothesis_id; reportv2 RC persist; promotion RC enforcement for family_id | phased_execution.md, testing_acceptance.md, risk_audit.md | Done | (PR link) | crypto_analyzer/db/migrations_phase3.py, crypto_analyzer/sweeps/hypothesis_id.py, crypto_analyzer/sweeps/store_sqlite.py, cli/research_report_v2.py, crypto_analyzer/promotion/service.py | test_migrations_phase3.py, test_hypothesis_id.py, test_sweep_store.py, test_promotion_service.py (sweep RC policy), test_reportv2_reality_check_optional.py (sweep persist) | run_migrations_phase3 idempotent; hypothesis_id order-invariant; reportv2 --reality-check + Phase 3 DB → sweep_families/sweep_hypotheses; family_id candidate requires RC for accept unless override | Opt-in: Phase 3 migrations only; default behavior unchanged |
| Phase 3 (Regimes slice): regime_runs / regime_states migrations (phase3 only, not default) | schema_plan.md | Done | crypto_analyzer/db/migrations_phase3.py | test_migrations_phase3*.py | run_migrations_phase3 only when ENABLE_REGIMES=1 + explicit opt-in | schema_migrations_phase3 table; not referenced by run_migrations() | |
| Phase 3 (Regimes slice): RegimeDetector (fit/predict, filter-only, no smoothing in test) | interfaces.md, risk_audit.md | Done | crypto_analyzer/regimes/regime_detector.py, regime_features.py | test_regime_detector.py, test_regime_features.py | mode="smooth" raises; probs sum to 1; causal features | threshold_vol with hysteresis | |
| Phase 3 (Regimes slice): regime materialize (stable run_id, regime_runs/regime_states) | schema_plan.md | Done | crypto_analyzer/regimes/regime_materialize.py | test_regime_materialize.py | materialize_regime_run gated by ENABLE_REGIMES=1 | Deterministic ts order; idempotent replace | |
| Phase 3 (Regimes slice): reportv2 --regimes optional (regime-conditioned IC summary) | phased_execution.md | Done | cli/research_report_v2.py | test_reportv2_regimes_optional.py | --regimes REGIME_RUN_ID only when ENABLE_REGIMES=1; default output unchanged | Slice 2: per-signal ic_summary_by_regime_, ic_decay_by_regime_, regime_coverage_*.json; exact join | |
| Phase 3 Slice 2: Regime-conditioned validation artifacts (per signal) + promotion gating (interfaces only) | phase3_regimes_slice2_alignment.md, testing_acceptance.md | Done | crypto_analyzer/validation/regime_conditioning.py, crypto_analyzer/promotion/gating.py, cli/research_report_v2.py, crypto_analyzer/validation_bundle.py | test_regime_conditioning_no_leakage.py, test_reportv2_regime_conditioned_artifacts.py, test_promotion_gating.py, test_reportv2_regimes_optional.py | Exact join on ts_utc; no leakage; require_regime_robustness=False default; ValidationBundle meta + path fields; evaluate_candidate deterministic. Regimes OFF: bundle JSON byte-identical; optional regime fields omitted when None. | phase3_regimes_slice2_alignment.md; run.ps1 verify | |
| Phase 3 Slice 3: Dynamic beta estimator (Kalman/RLS) in factor materialization | phase3_dynamic_beta_slice3_alignment.md, interfaces.md, schema_plan.md | Done | crypto_analyzer/factors_dynamic_beta.py, crypto_analyzer/factor_materialize.py | test_dynamic_beta_estimator.py, test_factor_materialize.py (kalman_beta tests) | estimator=kalman_beta; as_of_lag_bars>=1; same tables factor_model_runs/factor_betas/residual_returns; default rolling_ols unchanged | phase3_dynamic_beta_slice3_alignment.md; run.ps1 verify | |
| Phase 3 Slice 4: Reality Check (RC) + Romano–Wolf (implemented, opt-in); family_id; reportv2 --reality-check | phase3_reality_check_slice4_alignment.md, testing_acceptance.md, research_repo_mapping.md | Done | crypto_analyzer/sweeps/family_id.py, crypto_analyzer/stats/reality_check.py, cli/research_report_v2.py, crypto_analyzer/promotion/gating.py | test_family_id_stable.py, test_reality_check_deterministic.py, test_reality_check_null_sanity.py, test_reportv2_reality_check_optional.py | RC opt-in; family_id stable; stationary/block bootstrap null; registry keys family_id, rc_p_value, etc.; RW opt-in via CRYPTO_ANALYZER_ENABLE_ROMANOWOLF=1, outputs rw_adjusted_p_values when enabled | phase3_reality_check_slice4_alignment.md; run.ps1 verify | |
| Romano–Wolf (implemented, opt-in): CRYPTO_ANALYZER_ENABLE_ROMANOWOLF=1 | interfaces.md, methods_and_limits.md | Done | crypto_analyzer/stats/reality_check.py | test_reality_check_deterministic.py, test_reportv2_reality_check_optional.py | When env set, RW maxT stepdown runs; rw_adjusted_p_values in RC summary; stats_overview.json rw_enabled | Implementation-aligned; see methods_and_limits.md §9 | |
| Phase 3 Slice 5: Promotion workflow (exploratory→candidate→accepted); store + service + Streamlit + CLI; RC cache | phase3_promotion_slice5_alignment.md, phase3_slice5_pr_template.md, schema_plan.md, performance_scale.md | Done | crypto_analyzer/db/migrations_phase3.py, crypto_analyzer/promotion/store_sqlite.py, crypto_analyzer/promotion/service.py, crypto_analyzer/stats/rc_cache.py, cli/app.py, cli/promotion.py, cli/research_report_v2.py | test_migrations_phase3.py, test_promotion_store.py, test_promotion_service.py, test_promotion_e2e.py, test_rc_cache.py | promotion_candidates/promotion_events in phase3 migrations; list/create/evaluate CLI; Promotion page; RC null cache keyed by family_id+config+dataset+git; --no-cache; run_migrations_phase3 opt-in only | phase3_promotion_slice5_alignment.md, phase3_slice5_pr_template.md; run.ps1 verify | |
| Phase 3 PR2: Execution realism gates | phased_execution.md, risk_audit.md | Done | crypto_analyzer/promotion/execution_evidence.py, crypto_analyzer/promotion/gating.py, crypto_analyzer/promotion/service.py, cli/research_report_v2.py, cli/promotion.py, cli/app.py | test_execution_evidence.py, test_promotion_service.py, test_promotion_e2e.py | --execution-evidence in reportv2 writes capacity_curve + execution_evidence.json; evidence_json stores execution_evidence_path only; require_execution_evidence or target candidate/accepted enforces evidence unless allow_missing_execution_evidence (auditable); thresholds_used + warnings in event | run.ps1 verify | |
| Phase 3 PR3: Measured performance optimization | phased_execution.md, performance_scale.md | Done | crypto_analyzer/stats/cache_flags.py, crypto_analyzer/stats/factor_cache.py, crypto_analyzer/stats/regime_cache.py, crypto_analyzer/factor_materialize.py, crypto_analyzer/regimes/regime_materialize.py, crypto_analyzer/stats/rc_cache.py, cli/research_report_v2.py | test_factor_cache_hit_skips_compute.py, test_regime_cache_hit_skips_compute.py, test_profile_timings_optional.py | is_cache_disabled(no_cache_flag) single source of truth; factor/regime skip compute on hit (metadata + rowcount match); CRYPTO_ANALYZER_PROFILE=1 writes timings.json (in manifest only when profiling on); --no-cache wired to cache_flags | run.ps1 verify | |
| Slice 3: reportv2 optional --factor-run-id | architecture_simplification_plan.md, drift_register.md | Done | crypto_analyzer/data.py (load_factor_run), cli/research_report_v2.py | test_reportv2_factor_run_id_optional.py, test_load_factor_run_* | Optional --factor-run-id loads betas/residuals from factor_betas/residual_returns for mf_metrics; default unset = in-memory (no behavior change). Invalid run exits 1 with clear message. | Safe refactor; aligns reportv2 with materialized factor model. |
| Item | Value |
|---|---|
| Date | 2026-02-19 (PR2 + PR3 merged) |
| Scope | PR1 (sweep registry hardening), PR2 (execution realism gates), PR3 (measured performance optimization). All Phase 3 boxes in phased_execution.md checked. |
| Verify | Run .\scripts\run.ps1 verify; expect VERIFY PASS (doctor → pytest → ruff → research-only → diagrams). |
| Evidence | PR2: test_execution_evidence.py, test_promotion_service.py, test_promotion_e2e.py. PR3: test_factor_cache_hit_skips_compute.py, test_regime_cache_hit_skips_compute.py, test_profile_timings_optional.py. |
| Commit | 9809821ed33312fab11fd04d60277fa2ef842403 (stamped after final Phase 3 closeout). Post-closeout smoke fix: 00526c3b5ecb306675c969e384d4101aaef6961f (align signal shapes in orth corr report). |
Follow-up (done): reportv2 smoke --execution-evidence was failing when orthogonalization had ≥2 signals with different column counts; correlation-report block in signals_xs.orthogonalize_signals now uses _flatten_pair (intersection of columns + same shape) so only comparable cells are correlated. Fixed in 00526c3.
| Date | Decision | Rationale |
|---|---|---|
| (placeholder) | (e.g. Adopt BH at q=5% as default family-wide) | (optional) |
| (placeholder) |