Skip to content

test(query-engine): cover temporal anchors and topk ordering - #775

Merged
milindsrivastava1997 merged 3 commits into
mainfrom
771-testpromql-cover-nested-temporal-anchors-and-topk-ordering
Oct 4, 2026
Merged

milindsrivastava1997 merged 3 commits into
mainfrom
771-testpromql-cover-nested-temporal-anchors-and-topk-ordering

Conversation

@milindsrivastava1997

@milindsrivastava1997 milindsrivastava1997 commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Extends PromQL differential coverage for temporal functions and adds an opt-in instant-vector ordering policy for TopK/BottomK.

Test rules: before

  • Every vector and matrix response was normalized by label/timestamp before comparison.
  • /query, /query_range, and range-at-t/instant-at-t parity therefore compared result membership and values, but never response presentation order.
  • The aggregation suite covered rate, increase, sum_over_time, and count_over_time, plus selected direct aggregations.

Test rules: after

  • The default remains order-insensitive for ordinary instant vectors and all range matrices.
  • A query can opt into comparison.instant_vector_order with ascending or descending.
  • For an opted-in instant TopK/BottomK result, ASAPQuery must return values in the configured direction.
  • For by/without TopK/BottomK, each bucket must be contiguous and ordered internally; bucket order is intentionally unconstrained.
  • Equal-valued members may appear in any order, matching the Prometheus ordering rule.
  • /query_range and range-at-t/instant-at-t parity remain order-insensitive.

Coverage added

  • Native temporal cases for min_over_time, max_over_time, and valid sum/count/min/max temporal-reducer compositions across ungrouped, by, and without grouping.
  • Ordering policy on existing aggregation-suite TopK cases.
  • A dedicated tied-value TopK fixture covering ungrouped, by (job), and without (instance) behavior.
  • Runner unit coverage for policy parsing/validation, default unordered behavior, direction checking, bucket contiguity, grouped bucket reordering, and range ordering exclusion.

Docker compliance status

Suite Before this PR With this PR
single-rate-temporal Fail (pre-existing window mismatch) Fail (same mismatch)
single-rate-off-grid-rate Fail (pre-existing off-grid native-rate behavior) Fail (same behavior)
aggregations-native-dag Pass Pass
aggregations Fail (pre-existing unsupported rate shapes) Fail (also exposes unsupported avg_over_time)
topk-ordering N/A — new suite Fail: ungrouped ties pass; grouped by/without expose bucket interleaving
quantiles Pass Pass
olly-bench Expected non-CI failure Expected non-CI failure

Verification

  • go test ./... in promql-compliance/runner
  • Focused topk-ordering compliance run: ungrouped tied TopK passes; grouped cases expose the current engine bucket-interleaving bug.

Known follow-up work

  • Grouped native TopK currently interleaves buckets, so the grouped ordering checks remain red until the query-engine ordering fix lands.
  • avg_over_time and several nested/rate shapes currently return No result for query; nested temporal pipeline support is tracked in feat(planner): support nested temporal aggregation pipelines #772.

@milindsrivastava1997 milindsrivastava1997 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review: 3 findings.

}
return metricString(labels)
}
excluded := make(map[model.LabelName]struct{}, len(grouping.Labels))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without grouping keeps __name__, unlike Prometheus. Prometheus always drops __name__ from the grouping key for without, and topk/bottomk keep __name__ in their output, so a multi-metric selector gets split into per-name groups here.

Example: topk without (instance) (3, {__name__=~"a|b"}) → reference returns one group ordered a{i1}=5, b{i1}=4, a{i2}=3; the checker reports group "…a…" is not contiguous against a correct result. Conversely, a wrong cross-name order passes, since order is only checked within each name.

Suggest always excluding __name__ in the without branch.

name: topk-ordering

queries:
# Equal values make the full label set the TopK tie-breaker.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These cases can't catch a tie-breaking bug. topk(6, …) runs over exactly 6 series, and the by (job) / without (instance) cases with k=3 run over exactly 3 series per group, so every series is always selected and the tie-breaker never decides membership. compareInstantValues also doesn't check the order of tied samples. An engine that breaks ties differently would pass. Using k smaller than the group size would make these meaningful.

series:
- metric: ordered_data
labels: {job: backend, instance: a}
samples: [{offset_seconds: 0, value: 1}, {offset_seconds: 300, value: 1}, {offset_seconds: 600, value: 1}]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Samples end exactly at the last evaluated offset (600). aggregations.yaml deliberately adds a sample past the last evaluated offset so the trailing window has a real later sample to close on, rather than only the wall-clock idle fallback. Without one, the 600s evaluations here rely on that fallback and may be flaky in the ASAP engine. Consider adding a sample at e.g. 660.

@milindsrivastava1997

Copy link
Copy Markdown
Contributor Author

This was generated by AI during triage.

Triage outcome for the three findings:

  1. Fixed. without bucket keys now always exclude __name__, matching Prometheus grouping behavior. Commit 50a58c7 adds a two-metric regression test proving metric names cannot split one without (instance) bucket.

  2. Not applicable under the agreed semantics. Equal-value TopK members have no specified relative ordering or tie-selection rule. Reducing k would test an unspecified behavior. The fixture is for value ordering and grouped-bucket contiguity, so its comment now says that explicitly.

  3. Not applicable. These are bare-selector TopK queries, not temporal-window queries. Their offset-600 evaluation does not require a later sample to close a window; the focused run passes the ungrouped case at that point.

@milindsrivastava1997
milindsrivastava1997 merged commit 9554af5 into main Oct 4, 2026
2 checks passed
@milindsrivastava1997
milindsrivastava1997 deleted the 771-testpromql-cover-nested-temporal-anchors-and-topk-ordering branch October 4, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(promql): cover nested temporal anchors and topk ordering

1 participant