Repository navigation
Conversation
This was referenced Oct 4, 2026
…variant Stage 1 now keeps two forms of the workload (#509 Pass 2): the queries as written, and, when CSE merges something, the queries with identical sub-DAGs shared. Stage 3 treats the variant as an outer choice: the per-target dynamic program runs once per variant (the coupling guard also checks targets reading a common input in the shared variant) and the cheapest result wins. plan_selection::plan_stages runs Stage 1 -> 2 -> 3; the facade and the stage_pipeline devtool both call it, so the facade no longer pre-merges and the devtool shows the shared variants. A PromQL series has one sample per timestamp, so the identified scan now declares (series identity, ts) as a key, which CSE's legality rule needs to share the range selector. Example 1: 1 -> 48 -> 48 -> 1; P44 (P20's choices over a shared input) is selected at 52.201 vs 79.401. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/509-e1-pass2-identical
branch
from
October 5, 2026 06:21
41c0c78 to
a52409e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased on main d4869a7 (DF 54).
Why
#509 Stage 1 is Pass 1 (local alternatives) + Pass 2 (ASAP-aware CSE). Pass 2's identical-expression rule was missing. In Example 1 both queries read
http_requests_total[1m], so the doc's "Raw with a shared input" plan was never a candidate. The facade also calledshare_common_sub_dagsunconditionally while the devtool never did, so the two disagreed. In practice the facade's merge was a no-op for PromQL: the scan had no unique key, and CSE's legality rule refuses to share without one. Part of #580; stacked on #587 (#572).What
asap-logical-optimizer/src/pass2/identical_expressions.rs). Stage 1 returns sharing variants: Pass 1 over the queries as written, and Pass 1 over the queries with identical sub-DAGs merged by the existingshare_common_sub_dags. The shared variant is added only when the merge removes nodes. Shared candidates are labelled "… · shared input" and numbered after the independent ones.asap-plan-selection). The per-target DP runs once per variant, and the cheapest wins. In the shared variant, identical summary producers are also merged after composition. They are priced the same way in the DP, in exhaustive enumeration and in the final full check. The coupling guard also checks pairs of targets that read a common input. Fallback and the final full check apply per variant.plan_selection::plan_stagesruns Stage 1 (Pass 1 + Pass 2) → Stage 2 → Stage 3. The facade calls it and no longer pre-merges. The devtool calls it and only serializes the result (withdisplayset so every candidate is priced).with_promql_series_identitynow declares(series identity, ts)as the identified PromQL scan's unique key. A series has at most one sample per timestamp. CSE's legality rule needs this key to share the range selector.How it was checked
sum by (job) (rate(x[1m]))andmax by (job) (rate(x[1m])), 16 + 8 combinations, where a shared candidate wins.stage1_keeps_independent_and_shared_variantsandstage3_shared_input_is_not_costlier. Hydra was dropped from both loops; it stays covered by the still-ignoredstage1_q2_summary_families_are_heap_sketches_and_hydra, because Pass 1 has no Hydra alternative.stage3_selects_a_shared_input_plan: the selected plan shares the input, and it saves exactly one scan plus one range node.Before this PR (Example 1)
1 → 24 → 24 → 1. Selected P20 "Q1 exact (Sum acc, Rate acc) · Q2 exact (Sum acc)", 79.401 cpu ms. The scan (23.2) and range (4.0) are priced twice, once per query.
After this PR (Example 1)
1 → 48 → 48 → 1 (24 independent + 24 shared; 16 Count-Min + heap candidates invalid as before). Selected P44 "Q1 exact (Sum acc, Rate acc) · Q2 exact (Sum acc) · shared input", 52.201 cpu ms. The same choices as P20, with one scan → range:
The best CountSketch + heap plan is P48 at 164.201 (shared).
Gate
cargo fmt --all --check✓;cargo clippy --workspace --all-targets --all-features -- -D warnings✓;cargo test --workspace: 1,514 passed / 10 ignored (#587: 1,508 / 12). The difference is 4 new tests (2 Pass 2 unit tests,shared_target_dp_equals_exhaustive,stage3_selects_a_shared_input_plan) and 2 un-ignored. Viewer:python3 -m unittest discover -s tools/dag-viewer -p test_render.py, 29 OK (6 skipped).Refs #509, #580, #572.
🤖 Generated with Claude Code