eval: §6.3 rerun with one-scrape instant RQEs

[milindsrivastava1997]

Summary

Reruns §6.3 (autosketch-vs-asap-hydra) and the accuracy-target sweep (autosketch-vs-asap-targets) with #215's one-scrape instant RQEs. sum by (job) and quantile by (job) now look back 5 ms at 200/s, one sample per series (N = 1e3 per job), instead of 1 s (N = 2e5). The inputs are main's committed ones: #213's curves, cost table and Hydra study.

• Tables: re-exported with --keep-duplicate-queries. Only the 8 classic and all tables change, and only their 6 instant RQEs per metric (ids, e2e blocks and every other RQE are identical). The 4 multi-grouping tables are byte-identical, so their results stand.
• Reruns, for the 8 changed tables: --runs 3 --no-chosen results at 95%, plans (--runs 1, extracted), and the 90%/99% runs.
• AutoSketch benchmark times: reused. autosketch_benchmark_time.py found no new probes.
• Figures, summary_*.md and both READMEs regenerated.

Results (cheapest plan, CPU weights, vCPU)

Workload	ASAP	ASAP (no roll-ups)	PerQuery	AutoSketch	ASAP deployments
mixed	2.35 → 2.71	2.35 → 2.71	7.67	2,701	6 → 7
mixed, m = 16	37.6 → 43.4	37.6 → 43.4	123	43,221	96 → 112
mixed + multi-grouping	2.57 → 2.93	2.90 → 3.26	8.74	2,711	11 → 12
mixed + multi-grouping, m = 16	41.1 → 46.9	46.4 → 52.2	140	43,375	176 → 192
multi-grouping	unchanged

• ASAP rises ~15% on CPU and ~9% on Fargate, with +1 deployment per metric. sum by (job) (sum_over_time) used to share the instant 1 s exact-sum deployment. With the instant window at 5 ms it gets its own, which ingests the stream again (~0.36 vCPU per metric).
• PerQuery and AutoSketch are unchanged, since they never shared that deployment. So are the latency bounds, AutoSketch's probes, and the roll-up counts.
• No conclusion changes. ASAP stays 2.8–3.0× below PerQuery on mixed (1.7–3.0× at every SLA) and ~1,000× below AutoSketch. Roll-ups save 10% on mixed + multi-grouping (was 11%).
• Targets: 90% still equals 95%. At 99%, mixed is 2.90 and mixed + multi-grouping 3.53 vCPU. Every run has 0 RQEs dropped and 0 sanity violations; study: denser KLL/DDSketch grid and Hydra W 65536; §6.3 rerun #213's solver-tolerance violation on all-shared8 at 99% is gone.

Design decisions (please approve)

1. The multi-grouping results aren't rerun. Their tables are byte-identical, the same approach as the 2026-10-13 rerun.
2. AutoSketch benchmark times are reused. No new (sketch, config, shape) probe appeared, so the script measured nothing.
3. The reruns ran serially on a 16-core desktop, not idle cluster nodes. Costs are deterministic, so they're unaffected. Planning times of the mixed rows now come from a different machine than the multi-grouping rows; the README says to compare them only to about 2×. If the planning-time figure needs one machine, the 8 tables (~45 min serially here) should be rerun on a cluster node.

Testing

• The export reproduces the committed multi-grouping tables byte for byte. For classic and all it changes only the instant RQEs.
• All 8 changed tables: 0 RQEs dropped, every chosen accuracy measured.

🤖 Generated with Claude Code