Skip to content

feat: expose summary-maintenance lifecycle candidates for explicit binding - #476

Closed
zzylol wants to merge 12 commits into
feat/physical-alternatives-in-planspacefrom
feat/lifecycle-candidates
Closed

zzylol wants to merge 12 commits into
feat/physical-alternatives-in-planspacefrom
feat/lifecycle-candidates

Conversation

@zzylol

@zzylol zzylol commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #477.

Why

Planner is meant to expose legal summary-maintenance lifecycle candidates and
let the deployment price them over the whole workload (see
physical-planning-and-deployment.md). Today
plan_summary_maintenance_lifecycles enumerates the alternatives but also picks
one with Planner's own cost model, so ASAPQuery-backend cannot price or bind an
alternative Planner did not prefer. The lifecycle choice is what splits summary
state between precompute (ContinuouslyMaintained/Prepared/Shared) and
query time (Ephemeral).

What

Split lifecycle planning into enumerate then select:

  • enumerate_summary_maintenance_lifecycles(...) -> SummaryMaintenanceLifecycleCandidates:
    one entry per unique SummaryAgg, with all four lifecycles, their cost or
    rejection, and nothing selected.
  • SummaryMaintenanceLifecycleCandidates::guarantee(&lifecycle): the guarantee
    (mode, schedule) a choice would carry, so a caller can price before choosing.
  • SummaryMaintenanceLifecycleCandidates::select(&[(state, lifecycle)]): turns
    an explicit choice into the SummaryMaintenanceLifecyclePlan the physical
    compiler consumes. It refuses unknown or duplicate states, missing states,
    unlisted or rejected alternatives, incompatible nested schedules, and
    combinations the cost model cannot estimate (SummaryMaintenanceLifecycleChoiceError).

plan_summary_maintenance_lifecycles and the global-selection helpers now run
enumerate + cheapest-select through the same checks. No new cost logic; unknown
cost stays unknown.

Before this PR

let plan = plan_summary_maintenance_lifecycles(root, demand, now, horizon, caps, model)?;
// deployments[i] already carries Planner's chosen lifecycle.

After this PR

let candidates = enumerate_summary_maintenance_lifecycles(root, demand, now, horizon, caps, model)?;
// price candidates.deployments() in the deployment, then bind its choice:
let plan = candidates.select(&[(state_id, SummaryMaintenanceLifecycle::Ephemeral)])?;

Choosing what Planner would choose yields an identical plan.

Open point for review

An alternative whose only issue is missing cost evidence can be bound only when
the cost model declares its whole-combination hook authoritative for lifecycle
costs (Planner's existing rule). Accepting any legal-but-unpriced alternative is
a one-line change, but the plan's cost and window framework would then be
unknown. Left as is pending review.

The window framework is decided per whole combination by the cost model, so it
appears only in the plan returned by select, not per alternative.

Validation

6 new tests in summary_maintenance_lifecycle.rs (enumeration without
selection, reproducing Planner's choice, binding a costlier legal alternative,
illegal/incomplete choices, incompatible nested schedules, one entry per unique
state). cargo fmt --check and cargo clippy --workspace --all-targets -- -D warnings
pass; cargo test --workspace --no-fail-fast passes except the pre-existing #462
failure devtools dag_export::workload_node_ids_make_smaller_topk_reuse_explicit.

🤖 Generated with Claude Code

zzylol and others added 12 commits September 30, 2026 18:02
Start `asap-physical-operators` with thin summary kernels over
`asap_sketchlib`, the kernel capability checks and the typed value model.

The boundary is:
- `asap_sketchlib` owns sketch algorithms and their state encodings.
- Kernels hold one population's in-memory state. They expose `merge`, a
  typed sketch readout (`estimate(&SketchQuery)`) and memory accounting.
  Exact states answer a typed `ExactReadout`; empty MIN/MAX read as `None`.
- Group-by belongs to physical operators.
- Deployments own wire decoding, delta frames, edge sampling and storage
  statistics.

So wire decoding, `SerializableToSink`, `AggregationType`, `aux_stats`,
`reset_to_empty` and the keyed/sum/min/max kernels are not carried over from
ASAPQuery-backend. The `asap_sketch_codec` crate is not carried over either;
it moves to `asap_sketchlib`. Hydra KLL remains as the Hydra shared-grouping
kernel. HLL uses sketchlib's classic estimator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Rebased onto main: the Cargo.lock conflict with #478 (new asap-planner
crate) was resolved by regenerating workspace entries with
`cargo update -w --offline`.
Add the execution layer of `asap-physical-operators`:

- `plan`, `runtime` and `sources`: physical DAGs, run context, bounded
  backpressure, cancellation, memory reservations and raw-source scans.
- `expressions` and `operators`: relational/scalar operators, windows,
  temporal panes, current-series snapshots, and summary build, merge and
  readout. A readout is typed: a `SketchQuery`, or an `ExactReadout` whose
  counter lookback resolves to the run's evaluation range. Deserialized
  operators are validated before use.
- `readout`: readouts over merged exact summary states.

There is no physical planner yet; operators are built directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add `physical_planner`: reader-independent compilation of selected logical
candidates into precompute/query physical DAGs with typed materialization
frontiers, bounded frontier enumeration, workload cost selection, temporal
KLL pane compilation and PromQL row/value lowering. `dag` remains a
compatibility re-export.

Tests cover physical DAGs, plan recovery, precompute candidates and
populations, PromQL values and binaries, weighted TopK binding and
current-series heaps, plus Planner-to-execution integration tests. The design
doc describes the ownership boundary with deployments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pane geometry, pane population checks and per-pane scheduling are deployment
concerns. The physical layer keeps the computation the deployment binds:
per-input summary build, union, shared merge and readout.

- Remove `physical_planner::compile_temporal_pane_candidate` and its
  `TemporalPaneMaintenance`/`TemporalPaneCandidate`/`TemporalEntityIdentity`
  contract.
- Remove the `PaneInput` operator; `ScopeTimestamp` remains as a general
  operator in `operators/scope_timestamp`.
- Drop the `selected_temporal_lifecycle_compiles_panes_and_executes` E2E test
  and update the crate README and design doc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Planner search needs the physical row representation to propose whole-root
physical alternatives, and asap-aware-mapping cannot depend on the physical
runtime crate. promql_rows::with_series_identity keeps its signature and
delegates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend built one workload forest per per-query physical alternative by
calling SketchAlgorithmStrategy's proposal methods outside PlanSpace. Planner
now owns them: search_workload_with_targets asks each strategy's new
ReplacementStrategy::propose_for_root once per targeted root.
SketchAlgorithmStrategy resolves the series identity and proposes
current-series TopK, fixed-window and query-time Rate aggregation, and
realizations over per-series Rate state, finalized and deduplicated.

The candidates carry ReplacementProvenance::RootPhysicalRealization: DAG
assembly uses them verbatim, and global selection never commits them.
Existing candidates and their order are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PlanSpace decides what to compute; the summary maintenance lifecycle decides
node timing and the physical compiler reads it. The fixed-window and
query-time Rate aggregation candidates only rewrote a finalization node's
timing, so they are placement variants and no longer go through
propose_for_root. The backend still calls those methods directly.

SketchAlgorithmStrategy::propose_for_root now proposes only current-series
TopK heaps, which rank identity-carrying rows the logical root lacks. The
per-series Rate filter and the verbatim-assembly branch are removed: the
latter is unnecessary because non-Aggregate roots are already assembled
verbatim. RootPhysicalRealization stays so global selection never commits
an unvalidated identity-carrying readout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nding

Split plan_summary_maintenance_lifecycles into enumeration and selection so
a deployment can price every lifecycle alternative per unique summary state
and bind its own choice. Planner selection is unchanged: it now enumerates
and then selects the cheapest complete combination through the same path.

SummaryMaintenanceLifecycleCandidates::select validates that each choice is
an alternative Planner could select, enforces schedule compatibility, and
obtains window frameworks and totals from the same complete-candidate hook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the split/462-f-physical-planner branch from bb1abdd to 0ced125 Compare September 30, 2026 18:26
@zzylol
zzylol force-pushed the feat/lifecycle-candidates branch from 4bb8c86 to 07244ea Compare September 30, 2026 18:27
@zzylol
zzylol changed the base branch from split/462-f-physical-planner to feat/physical-alternatives-in-planspace September 30, 2026 18:27
@zzylol
zzylol force-pushed the feat/physical-alternatives-in-planspace branch 2 times, most recently from 5ccaa44 to 45c7fc6 Compare October 1, 2026 22:48
zzylol added a commit that referenced this pull request Oct 1, 2026
#482, #479, #485, #491)

Stage 2 materialization (#509): enumerate summary-maintenance lifecycle candidates, derive execution timing from the chosen lifecycle, compile once and cut per lifecycle, keep placement in the lifecycle layer, and plan maintained populations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol

zzylol commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Included in the squash merge of #491 into main (8b86af4).

@zzylol zzylol closed this Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant