feat(ir): define compatible logical summary merges

[zzylol]

Why

Builds on #645 (generic IR, CSE) and #567 (summary coverage), both merged. Extracts structural summary merge support from #555.

Before this PR: constructing ASAPOp::SummaryMerge fails because the operation is reserved.

After this PR: two structurally compatible KLL states can form a typed logical merge with no execution-phase assignment. Merge inputs must be nonempty, carry state with exactly one state field, and have identical family/parameter/grouping schemas. Empty, raw and mismatched states fail.

This PR is structural only and does not touch SummaryCoverage. Equal schemas cannot tell a KLL over latency from a KLL over size, and cannot show that the inputs cover disjoint rows. Both are decided by summary coverage, derived for every summary node by one SummaryCoverage::derive in #646, following the design in #573.

Schema/result-kind/state identity and regression tests are included. Runtime execution, materialization and derived timing belong to the later physical scopes. Subtract/delete remain reserved.

Key code interface

SummaryMerge is a logical ASAP operator whose inputs are existing operator nodes producing partial state:

pub enum ASAPOp {
    // Other variants omitted.
    SummaryMerge {
        children: Vec<Rc<OperatorNode>>,
    },
}

It uses the same node construction and validation APIs as other operators:

let merged = OperatorNode::new_shared(Operator::ASAP(
    ASAPOp::SummaryMerge {
        children: vec![pane_a, pane_b],
    },
))?;
merged.validate_structure()?;

Here pane_a and pane_b are Rc<OperatorNode> state producers. new_shared returns Result<Rc<OperatorNode>, SchemaDerivationError> and derives the output schema/result kind. validate_structure checks the complete reachable DAG, including the producers. No timing assignment is required.

The key ASAPOp methods are:

pub fn validate_inputs(&self) -> Result<(), SchemaDerivationError>;
pub fn output_schema(&self) -> Result<Schema, SchemaDerivationError>;
pub fn output_kind(&self) -> OperatorResultKind;
pub fn produced_state(&self) -> Option<&FieldDataType>;

For SummaryMerge:

• validate_inputs enforces the compatibility rules below.
• output_schema validates inputs, then clones the first input's complete schema.
• output_kind is OperatorResultKind::State.
• produced_state returns the non-plain field type from the first input. This is a metadata accessor, not a runtime merge or an independent validation step.

Grouped merge keeps groups apart: job='api' and job='worker' stay separate states. A later SummaryEstimate is the readout boundary that turns state into a plain value.

Requirements for merging two summaries

This PR enforces these structural requirements:

Requirement	Check
At least one input	An empty children list is rejected. The API is n-ary; a single compatible state is structurally allowed.
Every input produces state	Each child has result_kind == State; raw relations/vectors are rejected.
Exactly one state field	The first schema contains exactly one non-Plain field. Identical schemas enforce this on every other input too. Plain grouping fields may accompany it.
Same committed state type	Complete schema equality requires the same family, algorithm/kind, parameters and any grouping layout carried by FieldDataType. KLL k=200 and k=300, or KLL and CMS, cannot merge here.
Same grouping/output layout	Field positions, plain grouping-field types, names, qualifiers and nullability must match.
Same schema metadata	time_index, unique_keys and closed must match as well. Matching only the state algorithm is insufficient.
Structurally valid producers	Whole-DAG validate_structure checks each producer's own contracts.

Compatibility is deliberately strict: even differently named but otherwise equivalent schemas need an explicit normalization before this interface accepts them.

These checks establish typed structural compatibility only. They do not prove that the inputs summarize the same computation or cover disjoint rows (#646), that they cover the intended population/window, that a runtime implements the family's merge operation, or that the merged result meets an accuracy requirement. Those belong to #646, the logical composition rule and subsequent physical planning/selection. For example, overlapping frequency panes must not silently double-count observations; matching schemas alone cannot establish that.

When SummaryMerge can be used

During logical planning: a composition rule can construct SummaryMerge when it needs to combine compatible partial summary states—for example, several tumbling-window KLL panes answering one larger query window, or compatible partition summaries feeding a coarser computation. The planner must establish the intended input coverage and grouping semantics. The node can be constructed and structurally validated before choosing materialization.

During execution: a physical implementation can merge the state contents once its inputs are available and the selected family/runtime supports the operation. Timing follows the materialization choice in #509. For example, a query can merge previously ingested/stored panes, or merge states rebuilt for that query. #560 itself adds no runtime kernel, storage, retention, window generation or execution-phase policy, and does not imply runtime support for every declared state family.

Source: asap.rs, with regressions in crates/types/tests/summary_merge_structure.rs.

Validation: workspace tests, cargo fmt, and workspace/all-target Clippy with warnings denied all pass.

---

Base: main · Next: #646 · Tracker: #528

Order: #645 (merged) → #567 (merged) → #560 → #646 → #539 → #540 → #541 → #542 → #543.

🤖 Generated with Claude Code

[Selvomega]

Revised logical foundation 2/6 · Base: #567 · Next: #537 · Tracker: #528

Revised review order: #567 → #560 → #537 → #539 → #540 → #561.

🤖 Generated with Claude Code

This order seems to be stale

[Selvomega]

Looks weird. Why merged_coverage, something specific to SummaryMerge node only, will be a method for ASAPOp, a much broader type?

[zzylol]

Agreed, it only makes sense for SummaryMerge and had to return an error for every other operator. I replaced it with SummaryCoverage::of_merge(inputs) in summary_coverage.rs, which takes the merge's inputs directly; the three callers already have them from ASAPOp::SummaryMerge { children }.

[Selvomega]

No I still feel this unsolved. Please check my comments in code later

[zzylol]

Addressed in 42bc2f7 and e201d5a. merged_coverage and its replacement of_merge are both gone. ASAPOp has no coverage method, and this PR derives no coverage at all: the merged node carries none. SummaryMerge validation only reads each input's existing coverage, requiring it to be present (UnknownInput) and to have the same columns (ColumnMismatch, see below). Deriving coverage for every summary node, merges included, is one SummaryCoverage::derive in #646.

[Selvomega]

What does this function do?

[zzylol]

It returns what a summary state was built from: the update expression (input) and grouping (reduction) of the SummaryAgg that produced it. For a SummaryMerge it returns its first input's, which merge validation requires every input to share; for any other node it returns None.

SummaryMerge uses it to require that all inputs summarize the same expression with the same grouping. Equal schemas alone don't show that: KLL over latency and KLL over size, both grouped by job, have the same schema. I expanded the doc comment to say this.

[Selvomega]

Give me a few minutes to further understand this.

[Selvomega]

Figured out with Claude code. I feel the naming is off. I thought this is a function updating the summary

[zzylol]

This function is now removed (e201d5a). What it returned (the producing SummaryAgg's input and reduction) is part of SummaryCoverage instead, as its columns, next to the existing rows:

pub struct SummaryCoverage {
    pub source: Source,                 // rows
    pub regions: Vec<CoverageRegion>,   // rows: time × population
    pub input: SummaryUpdate,           // columns: = SummaryAgg.input (e.g. latency)
    pub group_by: Reduction,            // columns: = SummaryAgg.reduction (e.g. by job)
}

with_coverage rejects a SummaryAgg declaration whose columns differ from the node's own (ColumnMismatch), and a merge requires identical columns across inputs, so a KLL over latency and one over size (same schema) cannot merge.

[zzylol]

Yes, that was stale. The description now reads: #645 (merged) → #567 (merged) → #560 → #646 → #539 → #540 → #541 → #542 → #543.

[Selvomega]

This is still bad design. Since in #646 we introduced for_summary function to calculate the coverage, and that function is effectively only working for SummaryAgg nodes.

Taking an union of #646 and this PR on coverage deriving, we can see that we have

1. of_merge calculating coverage for SummaryMerge
2. for_summary calculating coverage for SummaryAgg
   This is very bad for the following 2 reasons
3. Ideally we should only have 1 function deriving coverage for all summary-generating nodes, where we have relatively consistent and unified behavior, now we have 2
4. Even for the current 2 in these PRs, neither their name nor their behavior match

I do hate the shitty patches agents are relentlessly making.

I propose here, we intendedly not implementing the coverage deriving functionality in this PR, since this is only to introduce the SummaryMerge node, and SummaryCoverage, by now, doesn't have any automatically inferred version. Then we do that in a unified way in #646, with one func

[zzylol]

Thanks for the review. This is a great idea.

[zzylol]

Done in 42bc2f7: this PR no longer derives or checks coverage. of_merge, UnknownInput and MergeOutputMismatch are removed, summary_coverage.rs matches main, and SummaryMerge only checks structure. #646 will derive coverage for every summary node through a single SummaryCoverage::derive(node, time_range).

[zzylol]

Follow-up in e201d5a: SummaryCoverage now also records the columns a state summarizes (input, group_by), so all merge compatibility beyond the schema goes through coverage. This PR still derives nothing; callers declare coverage, and with_coverage only checks the declared columns against the SummaryAgg's own fields. #646 derives rows and columns for SummaryAgg and SummaryMerge through one SummaryCoverage::derive. The design doc in #573 is updated to match.

[Selvomega]

Suggest renaming if this function is not used to actually update the summary.

[zzylol]

Renamed to summary_input_data in 4ff8b98. It reads the producing SummaryAgg's update expression (input, e.g. the latency column) and grouping (reduction, e.g. by job); it updates nothing. The doc comment now says this and that which rows were included is coverage, not this.

[zzylol]

Update: summary_input_data is removed in e201d5a; the same information is now SummaryCoverage.input and SummaryCoverage.group_by (see the thread above).