Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ members = [
"crates/types",
"crates/frontend-common",
"crates/sql-function-catalog",
"crates/asap-aware-mapping",
"crates/logical-optimizer",
"crates/physical-optimizer",
"crates/plan-selection",
Expand Down
14 changes: 0 additions & 14 deletions crates/asap-aware-mapping/Cargo.toml

This file was deleted.

30 changes: 0 additions & 30 deletions crates/asap-aware-mapping/src/lib.rs

This file was deleted.

2 changes: 1 addition & 1 deletion crates/devtools/src/bin/show_post_asap_ir.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// (or pipe via stdin: cargo run -p asap-devtools --bin show_post_asap_ir < queries.txt)
//
// Lowers a batch of ad-hoc SQL/PromQL queries to pre-ASAP IR, then runs the
// `asap-aware-mapping` pre-ASAP → post-ASAP binding pass and prints the
// `asap-logical-optimizer` pre-ASAP → post-ASAP binding pass and prints the
// resulting **post-ASAP IR** (the sketch-bound IR: an `OperatorNode` DAG in
// which `ASAPOp` operators — the concrete summary family/params committed per
// aggregate — replace the bound aggregates, while whatever the pass left
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ fn tally(corpus: &str) -> Tally {
}

/// Every query that lowers, additionally run through the pre-ASAP →
/// post-ASAP `asap-aware-mapping` binding pass (issue #98), at an
/// post-ASAP `asap-logical-optimizer` binding pass (issue #98), at an
/// approximate accuracy target so the sketch-selection boundary actually
/// fires (an `Exact` target would only ever exercise the exact-accumulator
/// arm).
Expand Down
8 changes: 4 additions & 4 deletions crates/integration-tests/tests/cse.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
//! Drives the full staged pipeline this issue lands: two independently
//! lowered `OperatorNode` DAGs → `share_common_sub_dags` (stage 1,
//! `asap-types::ir::cse`, run internally by `search_workload`) →
//! `search_workload` (stage 2, `asap-aware-mapping`) — and asserts the
//! `search_workload` (stage 2, `asap-logical-optimizer`) — and asserts the
//! sharing that stage 1 decides survives into stage 2's discovered
//! `CandidateLogicalASAPDAGs` as one genuinely shared `TargetSubDAGCandidates`, not just one shared
//! `Rc<OperatorNode>`. This is the "real caller" the issue's landing plan
Expand All @@ -15,9 +15,9 @@
//! Committing to one final, physically-materialized answer for a whole
//! workload (the former `implement_workload`/`implement_workload_with`,
//! which this test file used to drive instead of `search_workload`) is out
//! of `asap-aware-mapping`'s scope — see that crate's `lib.rs` `## Status`
//! of `asap-logical-optimizer`'s scope — see that crate's `lib.rs` `## Status`
//! section — so these tests assert on the discovered `CandidateLogicalASAPDAGs` shape
//! directly, the same way `asap-aware-mapping::replacement`'s own
//! directly, the same way `asap-logical-optimizer::pass1::replacement`'s own
//! `shared_aggregate_across_two_roots_gets_both_strategies_candidates` test
//! does, just exercised through the crate's public API from this external
//! integration-test crate.
Expand Down Expand Up @@ -70,7 +70,7 @@ fn duplicate_workload_queries_collapse_onto_one_memo_group() {
// TargetSubDAGCandidates with consumer_count 2 — ASAPStrategies's one
// ExactAggregate candidate *and* SharedSubDAGStrategy's share-vs-
// recompute pair, exactly as `shared_aggregate_across_two_roots_gets_both_strategies_candidates`
// (asap-aware-mapping::replacement's own equivalent, internal test)
// (asap-logical-optimizer::pass1::replacement's own equivalent, internal test)
// pins for the same fixture shape.
let group = space
.candidates_for_target(&space.roots[0].1)
Expand Down
2 changes: 1 addition & 1 deletion crates/logical-optimizer/src/accuracy/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ use asap_types::types::AccuracyTarget;

use crate::pass1::exact_composition::ExactOperation;

/// The deployment-extensible accuracy algebra. `asap-aware-mapping` ships
/// The deployment-extensible accuracy algebra. `asap-logical-optimizer` ships
/// [`DefaultAccuracyModel`]; a deployment with a proof for a composition the
/// default rejects (a registered cross-metric conversion, say) implements
/// this trait and passes it to
Expand Down
2 changes: 1 addition & 1 deletion crates/logical-optimizer/src/pass1/rollup.rs
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
//! own set through [`RollupStrategy::new`].
//! - **No materialized roll-up operator.** Actually building a pre-aggregated
//! summary/scan leaf at execution time is separate, larger work outside
//! `asap-aware-mapping`'s scope (see issue #254's own "Non-goal" section)
//! `asap-logical-optimizer`'s scope (see issue #254's own "Non-goal" section)
//! — this module only constructs the pre-ASAP `NonASAPOp::Aggregate`
//! rewrite; a `CostModel`/search engine decides whether to prefer it.
//! - **No cross-schema reconciliation** (see "`ColumnId` comparability"
Expand Down
2 changes: 1 addition & 1 deletion crates/logical-optimizer/tests/stage1_cost_independence.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ const FORBIDDEN_MODULES: &[&str] = &["cost_model", "recurrence"];
const FORBIDDEN_CRATES: &[&str] = &[
"asap-physical-optimizer",
"asap-plan-selection",
"asap-aware-mapping",
"asap-planner",
"asap-physical-operators",
];

Expand Down
2 changes: 1 addition & 1 deletion crates/physical-optimizer/tests/stage2_dependencies.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use std::path::Path;
/// Crates Stage 2 must not depend on: Stage 3, the facade and the executor.
const FORBIDDEN_CRATES: &[&str] = &[
"asap-plan-selection",
"asap-aware-mapping",
"asap-planner",
"asap-physical-operators",
];

Expand Down
6 changes: 1 addition & 5 deletions crates/plan-selection/tests/stage3_dependencies.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,7 @@
use std::path::Path;

/// Crates Stage 3 must not depend on: the facade and the executor.
const FORBIDDEN_CRATES: &[&str] = &[
"asap-aware-mapping",
"asap-planner",
"asap-physical-operators",
];
const FORBIDDEN_CRATES: &[&str] = &["asap-planner", "asap-physical-operators"];

/// The manifest names neither the facade nor the executor crate, so Cargo
/// rejects any import of them.
Expand Down
8 changes: 3 additions & 5 deletions crates/planner/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,11 @@ edition = "2021"
# The library facade (issue #429): one entry point that takes prepared input and
# returns the selected DAG. It is the only crate that depends on every frontend
# — before it, the sole facade re-exporting more than one was `asap-devtools`,
# a developer-tools crate. Everything below the frontend boundary lives in
# `asap-aware-mapping`, so a third party writing an optimization pass (issue
# #430) depends on that crate alone and never pulls in DataFusion.
# a developer-tools crate. It also owns the optimization pass (pass/, issue
# #430) that runs the #509 stage crates.
[dependencies]
asap-types = { path = "../types" }
asap-aware-mapping = { path = "../asap-aware-mapping" }
asap-logical-optimizer = { path = "../logical-optimizer" }
asap-plan-selection = { path = "../plan-selection" }
asap-frontend-sql = { path = "../frontend-sql" }
asap-frontend-promql = { path = "../frontend-promql" }
Expand All @@ -22,4 +21,3 @@ thiserror = "2"
# The SQL frontend plans through DataFusion, which is async; `e2e_plan` is
# therefore async and its tests need a runtime.
tokio = { version = "1", features = ["rt", "macros", "rt-multi-thread"] }
asap-logical-optimizer = { path = "../logical-optimizer" }
15 changes: 8 additions & 7 deletions crates/planner/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,12 @@
//!
//! ```text
//! PlanningWorkload ──lowering──▶ ParsedWorkload ──optimization pass──▶ PlanOutput
//! (this crate) (asap-aware-mapping)
//! (front ends) ([`pass`]: #509 Stages 1–3)
//! ```
//!
//! [`e2e_plan`] runs both stages. A caller who already holds pre-ASAP IR — a
//! new frontend, a deserialized plan, a test that does not want to build SQL
//! and a catalog — skips this crate and calls
//! [`asap_aware_mapping::optimize`] directly.
//! and a catalog — calls [`optimize`] directly.

use asap_types::workload::parsed_workload::{ParsedWorkload, ParsedWorkloadError};
use asap_types::workload::{PlanningWorkload, QueryLanguage, SqlDialect, WorkloadError};
Expand All @@ -21,14 +20,16 @@ use asap_frontend_promql::{
};
use asap_frontend_sql::{lower_sql_dialect, SqlCatalog, SqlError};

pub mod pass;

// The optimization stage's vocabulary is this facade's vocabulary too: a caller
// configures the same models and reads the same output whether it goes through
// `e2e_plan` or straight to `optimize`.
pub use asap_aware_mapping::pass::{
optimize, OptimizationInput, OptimizationPass, OptimizeError, PassRegistry, PlanOutput,
QueryPlan, StagePipeline,
};
pub use asap_plan_selection::PlanningModels;
pub use pass::{
optimize, OptimizationInput, OptimizationInputError, OptimizationPass, OptimizeError,
PassNameConflict, PassRegistry, PlanOutput, QueryPlan, StagePipeline,
};

// ── Input ────────────────────────────────────────────────────────────────

Expand Down
4 changes: 2 additions & 2 deletions crates/planner/tests/e2e_plan.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@

use std::rc::Rc;

use asap_aware_mapping::pass::{OptimizationInput, OptimizationPass, OptimizeError, PlanOutput};
use asap_frontend_sql::{lower_sql_dialect, SqlCatalog};
use asap_logical_optimizer::pass1::logical_candidates::enumerate_local_logical_candidates;
use asap_plan_selection::{select_exhaustive, PlanningModels, MAX_ENUMERATED_CANDIDATES};
use asap_planner::pass::{OptimizationInput, OptimizationPass, OptimizeError, PlanOutput};
use asap_planner::{e2e_plan, FrontendInput, PlanError, UserInput, UserInputError};
use asap_types::ir::schema::{DataType, Field, Schema};
use asap_types::ir::QueryRoot;
Expand Down Expand Up @@ -242,7 +242,7 @@ async fn harness_rejects_a_pass_that_mislabels_entry_indices() {
"mangling"
}
fn optimize(&self, input: OptimizationInput<'_>) -> Result<PlanOutput, OptimizeError> {
let mut output = asap_aware_mapping::StagePipeline.optimize(input)?;
let mut output = asap_planner::StagePipeline.optimize(input)?;
for plan in output.plans.iter_mut() {
plan.entry_index += 1;
}
Expand Down
2 changes: 1 addition & 1 deletion crates/planner/tests/summary_sharing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ use asap_types::ir::cse::share_common_sub_dags;
use asap_types::ir::{ASAPOp, OperatorNode};
use std::rc::Rc;

use asap_aware_mapping::pass::{PlanOutput, QueryPlan};
use asap_frontend_promql::lower_promql_workload;
use asap_frontend_sql::SqlCatalog;
use asap_logical_optimizer::accuracy::{
Expand All @@ -19,6 +18,7 @@ use asap_logical_optimizer::{
use asap_plan_selection::candidate_selection::global_selection;
use asap_plan_selection::PlanningModels;
use asap_plan_selection::{CostModel, DefaultCostModel};
use asap_planner::pass::{PlanOutput, QueryPlan};
use asap_planner::{e2e_plan, FrontendInput, UserInput};
use asap_types::ir::operator::agg_intent::default_quantile;
use asap_types::ir::operator::AggIntent;
Expand Down
12 changes: 6 additions & 6 deletions crates/types/src/dag_export.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
//! out of [`export`]. It exists so a *higher* layer — one that depends on
//! `asap_types`, never the reverse — can annotate an already-exported DAG
//! after the fact without this module needing to know anything about that
//! layer's concepts. Concretely: `asap-aware-mapping`'s `explanation` module
//! layer's concepts. Concretely: `asap-logical-optimizer`'s `explanation` module
//! (issue #257) computes `structural_hash` over the same nodes this module
//! does (via the identical function). The devtools exporter uses that hash
//! to narrow candidates, then compares its target with
Expand Down Expand Up @@ -118,7 +118,7 @@ pub struct DAGNode {
/// Arbitrary reporting-layer annotations for this node — e.g. why a
/// replacement exists here. `asap_types` never populates this itself
/// (it has no notion of a "replacement" at all — see the module doc's
/// layering note); a higher layer that does (`asap-aware-mapping`, via
/// layering note); a higher layer that does (`asap-logical-optimizer`, via
/// the `dag_export` devtools binary) fills it in after the fact by
/// matching [`DAGNode::hash`] and confirming structural equality. Empty
/// by default, so every existing [`export`] caller and test is unaffected.
Expand All @@ -134,7 +134,7 @@ pub struct DAGNode {
/// One reporting-layer annotation attached to a [`DAGNode`] by a higher
/// layer than `asap_types` — see [`DAGNode::notes`]. `asap_types` defines
/// this shape (so the field has a concrete, serializable type) but never
/// constructs one: `asap_types` is a lower crate that `asap-aware-mapping`
/// constructs one: `asap_types` is a lower crate that `asap-logical-optimizer`
/// depends on, never the reverse, so this type is deliberately generic and
/// crate-agnostic rather than naming anything from that higher layer (e.g.
/// its `ExplanationKind`/`ReplacementExplanation`).
Expand Down Expand Up @@ -222,7 +222,7 @@ pub struct NamedDAG {
/// Concrete post-ASAP replacement sites discovered for this query — see
/// [`TargetReplacement`]. Always empty coming out of anything in this
/// module (same layering rule as [`DAGNode::notes`]: `asap_types` never
/// runs `asap-aware-mapping`'s search itself); a higher layer populates
/// runs `asap-logical-optimizer`'s search itself); a higher layer populates
/// this after the fact, e.g. the `dag_export` devtools binary's
/// `--post-asap` flag. Omitted from the JSON entirely when empty, so
/// every existing producer/consumer of `NamedDAG` (in particular every
Expand Down Expand Up @@ -290,7 +290,7 @@ pub struct WorkloadDAG {
// ── Post-ASAP replacement export — a layering-seam-shaped feature ──────────
//
// [`TargetReplacement`] is the generic, crate-agnostic "one replacement
// site, before and after" shape a higher layer (`asap-aware-mapping`, via
// site, before and after" shape a higher layer (`asap-logical-optimizer`, via
// the `dag_export` devtools binary's `--post-asap` flag) populates after
// running its own search — the exact same layering rule [`DAGNode::notes`]'s
// doc above already states: this module never runs
Expand All @@ -299,7 +299,7 @@ pub struct WorkloadDAG {
// `ReplacementProvenance` or a cost model even is. It only defines shapes
// concrete and serializable enough for a higher layer to fill in, and for
// `tools/dag-viewer` to render without needing to know anything about
// `asap-aware-mapping`'s own vocabulary.
// `asap-logical-optimizer`'s own vocabulary.

/// One flattened node of a [`SummaryDAG`] — the same node as a
/// [`DAGNode`], in the shape the summary-maintenance consumers read:
Expand Down
3 changes: 2 additions & 1 deletion docs/design_docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ an unsupported physical alternative into a deployable plan.
| Shared IR | `asap-types` | The unified operator IR (`ir`: one `OperatorNode` before and after ASAP optimization), schemas, workloads, guarantees, and exported plan data |
| Front-end common | `frontend-common` | Name-based `UnresolvedOp` tree shared by the front ends, and `resolve_root` into the operator IR |
| Query frontends | `frontend-sql`, `frontend-promql`, `frontend-metricsql` | Parse source languages and produce canonical Pre-ASAP queries |
| ASAP-aware mapping | `asap-aware-mapping` | Candidate generation, CSE, legality, accuracy propagation, costing, and ranking |
| ASAP-aware mapping | `asap-logical-optimizer`, `asap-physical-optimizer`, `asap-plan-selection` | #509 Stages 1–3: candidate generation, CSE, legality and accuracy propagation; physical candidates; costing and selection |
| Planner facade | `asap-planner` | Lowering dispatch, the optimization pass (`OptimizationPass`, `StagePipeline`) and `optimize` |
| Developer inspection | `devtools` | Expose planner DAGs, alternatives, decisions, and explanations for inspection |
| End-to-end validation | `integration-tests` | Verify behavior across frontends, mapping, and output IR |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -224,21 +224,20 @@ pass no longer runs these steps; another pass need not run any of them.
| Crate | What it holds |
|---|---|
| `asap-types` | `ParsedWorkload` |
| `asap-aware-mapping` | `OptimizationPass`, `OptimizationInput`, `PlanOutput`, `PlanningModels`, `optimize`, `PassRegistry`, `StagePipeline` |
| `asap-planner` *(new)* | `e2e_plan`, `UserInput`, `FrontendInput`, lowering dispatch |
| `asap-plan-selection` | `PlanningModels` |
| `asap-planner` | `e2e_plan`, `UserInput`, `FrontendInput`, lowering dispatch; `OptimizationPass`, `OptimizationInput`, `PlanOutput`, `optimize`, `PassRegistry`, `StagePipeline` |

```text
asap-planner ──┬──> asap-frontend-{sql, promql, metricsql}
└──> asap-aware-mapping ──> asap-types
▲
a pass depends only this far
└──> asap-plan-selection ──> asap-physical-optimizer ──> asap-logical-optimizer ──> asap-types
```

`asap-planner` is separate because it is the only crate depending on every
frontend; before it, the sole facade re-exporting more than one was
`asap-devtools`, a developer-tools crate. `PlanningModels` lives in
`asap-aware-mapping` (`plan_selection`) because both inputs use it, and
`asap-planner` re-exports it.
`asap-plan-selection` because both inputs use it, and `asap-planner`
re-exports it. Since #572 the pass lives in `asap-planner`, so a pass
implementation depends on the frontends too.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/design_docs/concepts/accuracy-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ rule or a different estimator configuration would be needed in those cases.

## Organization and extension contract

The `asap-aware-mapping::accuracy` module separates these responsibilities:
The `asap-logical-optimizer::accuracy` module separates these responsibilities:

```text
accuracy/
Expand Down
2 changes: 1 addition & 1 deletion docs/design_docs/decisions/concat-unique-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ for why it's fine to ship unused.
caller upstream of `resolve_root`, even though no such caller exists yet.
- Every other match/construction site touching `Concat` across the DAG
(`canonicalize.rs`, `cse.rs`, `schema_resolver.rs`, `dag_export.rs`,
`asap-aware-mapping`'s `replacement.rs`/`explanation.rs`, and every
`asap-logical-optimizer`'s `replacement.rs`/`explanation.rs`, and every
test/tooling AST walker) was mechanically updated to bind or ignore the new
field — most just added `, ..`; the two places that *rebuild* a `Concat`
node (`cse.rs`'s `rebuild_children`, part of CSE interning) thread
Expand Down
Loading