Repository navigation
Conversation
zzylol
force-pushed
the
feat/lifecycle-candidates
branch
from
September 30, 2026 02:14
59b41bd to
4bb8c86
Compare
zzylol
changed the base branch from
feat/shared-physical-operators
to
split/462-f-physical-planner
September 30, 2026 02:14
This was referenced Sep 30, 2026
Start `asap-physical-operators` with thin summary kernels over `asap_sketchlib`, the kernel capability checks and the typed value model. The boundary is: - `asap_sketchlib` owns sketch algorithms and their state encodings. - Kernels hold one population's in-memory state. They expose `merge`, a typed sketch readout (`estimate(&SketchQuery)`) and memory accounting. Exact states answer a typed `ExactReadout`; empty MIN/MAX read as `None`. - Group-by belongs to physical operators. - Deployments own wire decoding, delta frames, edge sampling and storage statistics. So wire decoding, `SerializableToSink`, `AggregationType`, `aux_stats`, `reset_to_empty` and the keyed/sum/min/max kernels are not carried over from ASAPQuery-backend. The `asap_sketch_codec` crate is not carried over either; it moves to `asap_sketchlib`. Hydra KLL remains as the Hydra shared-grouping kernel. HLL uses sketchlib's classic estimator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Rebased onto main: the Cargo.lock conflict with #478 (new asap-planner crate) was resolved by regenerating workspace entries with `cargo update -w --offline`.
Add the execution layer of `asap-physical-operators`: - `plan`, `runtime` and `sources`: physical DAGs, run context, bounded backpressure, cancellation, memory reservations and raw-source scans. - `expressions` and `operators`: relational/scalar operators, windows, temporal panes, current-series snapshots, and summary build, merge and readout. A readout is typed: a `SketchQuery`, or an `ExactReadout` whose counter lookback resolves to the run's evaluation range. Deserialized operators are validated before use. - `readout`: readouts over merged exact summary states. There is no physical planner yet; operators are built directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add `physical_planner`: reader-independent compilation of selected logical candidates into precompute/query physical DAGs with typed materialization frontiers, bounded frontier enumeration, workload cost selection, temporal KLL pane compilation and PromQL row/value lowering. `dag` remains a compatibility re-export. Tests cover physical DAGs, plan recovery, precompute candidates and populations, PromQL values and binaries, weighted TopK binding and current-series heaps, plus Planner-to-execution integration tests. The design doc describes the ownership boundary with deployments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pane geometry, pane population checks and per-pane scheduling are deployment concerns. The physical layer keeps the computation the deployment binds: per-input summary build, union, shared merge and readout. - Remove `physical_planner::compile_temporal_pane_candidate` and its `TemporalPaneMaintenance`/`TemporalPaneCandidate`/`TemporalEntityIdentity` contract. - Remove the `PaneInput` operator; `ScopeTimestamp` remains as a general operator in `operators/scope_timestamp`. - Drop the `selected_temporal_lifecycle_compiles_panes_and_executes` E2E test and update the crate README and design doc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Planner search needs the physical row representation to propose whole-root physical alternatives, and asap-aware-mapping cannot depend on the physical runtime crate. promql_rows::with_series_identity keeps its signature and delegates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend built one workload forest per per-query physical alternative by calling SketchAlgorithmStrategy's proposal methods outside PlanSpace. Planner now owns them: search_workload_with_targets asks each strategy's new ReplacementStrategy::propose_for_root once per targeted root. SketchAlgorithmStrategy resolves the series identity and proposes current-series TopK, fixed-window and query-time Rate aggregation, and realizations over per-series Rate state, finalized and deduplicated. The candidates carry ReplacementProvenance::RootPhysicalRealization: DAG assembly uses them verbatim, and global selection never commits them. Existing candidates and their order are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PlanSpace decides what to compute; the summary maintenance lifecycle decides node timing and the physical compiler reads it. The fixed-window and query-time Rate aggregation candidates only rewrote a finalization node's timing, so they are placement variants and no longer go through propose_for_root. The backend still calls those methods directly. SketchAlgorithmStrategy::propose_for_root now proposes only current-series TopK heaps, which rank identity-carrying rows the logical root lacks. The per-series Rate filter and the verbatim-assembly branch are removed: the latter is unnecessary because non-Aggregate roots are already assembled verbatim. RootPhysicalRealization stays so global selection never commits an unvalidated identity-carrying readout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nding Split plan_summary_maintenance_lifecycles into enumeration and selection so a deployment can price every lifecycle alternative per unique summary state and bind its own choice. Planner selection is unchanged: it now enumerates and then selects the cheapest complete combination through the same path. SummaryMaintenanceLifecycleCandidates::select validates that each choice is an alternative Planner could select, enforces schedule compatibility, and obtains window frameworks and totals from the same complete-candidate hook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
split/462-f-physical-planner
branch
from
September 30, 2026 18:26
bb1abdd to
0ced125
Compare
zzylol
force-pushed
the
feat/lifecycle-candidates
branch
from
September 30, 2026 18:27
4bb8c86 to
07244ea
Compare
zzylol
changed the base branch from
split/462-f-physical-planner
to
feat/physical-alternatives-in-planspace
September 30, 2026 18:27
zzylol
force-pushed
the
feat/physical-alternatives-in-planspace
branch
2 times, most recently
from
October 1, 2026 22:48
5ccaa44 to
45c7fc6
Compare
zzylol
added a commit
that referenced
this pull request
Oct 1, 2026
#482, #479, #485, #491) Stage 2 materialization (#509): enumerate summary-maintenance lifecycle candidates, derive execution timing from the chosen lifecycle, compile once and cut per lifecycle, keep placement in the lifecycle layer, and plan maintained populations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #477.
Why
Planner is meant to expose legal summary-maintenance lifecycle candidates and
let the deployment price them over the whole workload (see
physical-planning-and-deployment.md). Todayplan_summary_maintenance_lifecyclesenumerates the alternatives but also picksone with Planner's own cost model, so ASAPQuery-backend cannot price or bind an
alternative Planner did not prefer. The lifecycle choice is what splits summary
state between precompute (
ContinuouslyMaintained/Prepared/Shared) andquery time (
Ephemeral).What
Split lifecycle planning into enumerate then select:
enumerate_summary_maintenance_lifecycles(...) -> SummaryMaintenanceLifecycleCandidates:one entry per unique
SummaryAgg, with all four lifecycles, their cost orrejection, and nothing selected.
SummaryMaintenanceLifecycleCandidates::guarantee(&lifecycle): the guarantee(mode, schedule) a choice would carry, so a caller can price before choosing.
SummaryMaintenanceLifecycleCandidates::select(&[(state, lifecycle)]): turnsan explicit choice into the
SummaryMaintenanceLifecyclePlanthe physicalcompiler consumes. It refuses unknown or duplicate states, missing states,
unlisted or rejected alternatives, incompatible nested schedules, and
combinations the cost model cannot estimate (
SummaryMaintenanceLifecycleChoiceError).plan_summary_maintenance_lifecyclesand the global-selection helpers now runenumerate + cheapest-select through the same checks. No new cost logic; unknown
cost stays unknown.
Before this PR
After this PR
Choosing what Planner would choose yields an identical plan.
Open point for review
An alternative whose only issue is missing cost evidence can be bound only when
the cost model declares its whole-combination hook authoritative for lifecycle
costs (Planner's existing rule). Accepting any legal-but-unpriced alternative is
a one-line change, but the plan's cost and window framework would then be
unknown. Left as is pending review.
The window framework is decided per whole combination by the cost model, so it
appears only in the plan returned by
select, not per alternative.Validation
6 new tests in
summary_maintenance_lifecycle.rs(enumeration withoutselection, reproducing Planner's choice, binding a costlier legal alternative,
illegal/incomplete choices, incompatible nested schedules, one entry per unique
state).
cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warningspass;
cargo test --workspace --no-fail-fastpasses except the pre-existing #462failure
devtools dag_export::workload_node_ids_make_smaller_topk_reuse_explicit.🤖 Generated with Claude Code