Skip to content

refactor!: align DAG APIs and share candidate graphs - #480

Open
zzylol wants to merge 19 commits into
refactor/candidate-post-asap-dagsfrom
docs/planner-output-layers
Open

zzylol wants to merge 19 commits into
refactor/candidate-post-asap-dagsfrom
docs/planner-output-layers

Conversation

@zzylol

@zzylol zzylol commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #508.

Rebuilt into the linear stack on main. The 12 docs commits before the former build: merge were written against main; each resolves docs/design_docs/architecture/input-output-workflow.md to that commit's version, as integration merge e59640f did. That merge is no longer needed. As in e59640f, this PR's text of that file replaces the #485 sentence about lifecycle-assigned timing and the #508 renames there.

Why

The architecture's DAG names did not identify the Rust APIs, and callers had to compose a separate lifecycle enumerator and physical candidate wrappers between layers. The public pipeline now uses named candidate collections end to end, with shared logical graphs and physical compilation.

What

  • Stack on refactor: rename PlanSpace to CandidatePostASAPDAGs #508 and retain the workload facade already on main.
  • Align individual graph names: PreASAPDAG, PostASAPDAG, PhysicalDAG; distinguish nodes and runtime-bound BoundPhysicalDAG. Remove compatibility aliases for the old names.
  • Expose lower_pre_asap_dag_candidates returning CandidatePreASAPDAGs with normalized workload entry IDs.
  • Give CandidatePostASAPDAGs a timed stage. with_timing_for_root encapsulates logical realization, lifecycle enumeration, shared indexing, assignment budgets and lazy iteration. Existing assembled graphs use from_post_asap_dag. The lifecycle enumerator is now crate-private.
  • Make compile_physical_dag_candidates directly consume the timed logical collection and return CandidatePhysicalDAGs<Id>. The physical collection owns shared compiled graphs and cut descriptors; the public PhysicalDAGCandidate wrapper and tuple-based compilation API are removed.
  • Preserve workload/candidate IDs, lifecycle metadata, logical assembly rejections, timing failures and compilation errors. Unknown costs stay unknown.
  • Reuse physical compilation only when logical identity, Binary timing, input contracts and requested roots agree. Materialize typed precompute/query cuts on demand.
  • Reuse the existing lifecycle enumeration, graph validator, operator compiler and cut implementation. Explicit selection helpers remain available.
  • Update the implementation plan, DAG naming table and breaking API migration guide.

Before this PR

A deployment used Rc<QueryExpr>, Rc<SummaryNode>, PostAsapDag and CompiledPhysicalDag for graph stages. It separately enumerated lifecycle choices, attached timing and assembled physical candidate wrappers.

After this PR

CandidatePreASAPDAGs → CandidatePostASAPDAGs → CandidatePostASAPDAGs with timing → CandidatePhysicalDAGs

A caller invokes logical.with_timing_for_root(...), then passes the resulting collection directly to compile_physical_dag_candidates(...). It can inspect physical graphs and diagnostics through iter() and obtain typed execution cuts with materialize(index). Compatible assignments share one compiled graph; no intermediate stage selects a winner.

Generation can be scoped to a workload root without expanding independent query combinations. Deployment selection must still coordinate shared state across roots. Missing window evidence must be resolved before installation. Downstream consumers must migrate the breaking API names before repinning.

Validation

  • cargo test --workspace: 1509 passed, 0 failed, across 91 test groups.
  • cargo clippy --workspace --all-targets -- -D warnings and formatting checks passed.
  • Regressions cover collection-to-collection handoff, workload entry identity, lazy assignment budgets, unknown costs, graph sharing, Binary timing, contract/root-sensitive compilation reuse, transport equivalence and error preservation.
  • Alignment documentation links/fences and git diff --check passed.

🤖 Generated with Claude Code

@zzylol zzylol changed the title docs: define how PlanSpace, PlanOutput, and PostAsapDag relate docs: define Planner DAG names and planning layers Sep 30, 2026
@zzylol zzylol changed the title docs: define Planner DAG names and planning layers docs: define DAG names and preserve candidates through Planner layers Sep 30, 2026
@zzylol zzylol changed the title docs: define DAG names and preserve candidates through Planner layers refactor!: align DAG APIs and share candidate graphs Sep 30, 2026
@zzylol
zzylol changed the base branch from main to refactor/candidate-post-asap-dags September 30, 2026 16:56
zzylol and others added 19 commits September 30, 2026 18:18
#445, #470, #472, and #478 each described the Planner output from a
different angle. Add an output-layers section that places them in order:
candidate space, selected logical plan, and exported logical DAG. Say that
PlanOutput is derived from PlanSpace rather than being a second output.
Use "candidate" instead of "alternative" throughout input-output-workflow.md.

Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al, deployment

Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record the owner-approved layering: logical PlanSpace, summary lifecycle as
the only source of timing, physical compilation cut by timing, and a
deployment that prices lifecycle assignments, supplies data and state, and
executes Planner-compiled DAGs. Link it from the output layers section.

Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use PreASAPDAG, PostASAPDAG, and PhysicalDAG as design names while mapping current Rust APIs. Document frontend lowering, logical candidates, lifecycle-only timing, and physical compilation/cuts.

Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use candidate PostASAPDAGs as the design name and CandidatePostASAPDAGs for the renamed Rust collection. Keep explicit selection helpers separate from the candidate-preserving pipeline.

Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebase note: conflicts in docs/design_docs/architecture/input-output-workflow.md
with the stack below are resolved to this commit's version of the file, as
integration merge e59640f resolved them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Name shared DAG roots and frontend candidate collections; distinguish bound execution graphs from compiled PhysicalDAGs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compile shared logical roots through the existing validator and operator lowering. Enumerate lifecycle assignments without implicit winner selection, retain unknown costs and rejections, and share compatible compiled physical graphs across on-demand timing cuts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Encapsulate lifecycle enumeration in CandidatePostASAPDAGs with timing. Compile that collection directly into CandidatePhysicalDAGs, retaining metadata and failures while sharing compatible compilations. Remove public lifecycle enumerator and physical candidate descriptor APIs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the refactor/candidate-post-asap-dags branch from ea12b31 to 763bef3 Compare September 30, 2026 18:27
@zzylol
zzylol force-pushed the docs/planner-output-layers branch from 5307489 to 536dabf Compare September 30, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants