Add HPK test coverage for Cosmos#4682
Open
simorenoh wants to merge 13 commits into
Open
Conversation
Add a dedicated cosmos_hpk emulator test module plus hierarchical partition key (HPK) negative validation tests in cosmos_feed_ranges, closing the end-to-end HPK gaps tracked in #4155. Covers full-key item CRUD, patch, full-key query scope, numeric/bool/ empty-string components, transactional batch (happy path and 207 mismatch), partial-key/too-many-component point ops, and feed-range validation (empty / too-many / multi-on-single-hash). Five tests are marked #[ignore] documenting verified product gaps: prefix queries are not filtered server-side (#4680) and cross-partition queries over HPK containers fail with 400 (#4681). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This is a test-only PR that closes the end-to-end Hierarchical Partition Key (HPK) coverage gap tracked in #4155. It adds a new emulator_tests::cosmos_hpk module exercising the public azure_data_cosmos surface against MultiHash (2- and 3-level) containers, and extends cosmos_feed_ranges with HPK client-side negative-validation tests. Five tests that surface known SDK gaps (prefix-query filtering #4680, cross-partition fan-out over HPK containers #4681) are written but #[ignore]d so they can be flipped on once those issues are fixed. No library code is modified.
Changes:
- New
cosmos_hpkmodule: point CRUD/patch/upsert/delete, partial-key & too-many-component rejection, full-key/prefix/cross-partition queries, component value edge cases, and transactional batch (happy + PK-mismatch). - Extended
cosmos_feed_rangeswith a "Group C" set locking in the exactCosmosStatusreturned for empty / too-many-component / single-hash-multi-component keys. - Registered the module in
mod.rsand added an "Other Changes" CHANGELOG entry.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_hpk.rs |
New HPK emulator test module; solid coverage, but several doc comments overstate/misdescribe behavior (claimed null coverage, prefix-filtering claim, collect_query "all tests" claim, "single-hash" label). |
sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_feed_ranges.rs |
Adds Group C HPK feed-range client-side validation tests; verified against feed_range_from_partition_key and CosmosStatus constants. |
sdk/cosmos/azure_data_cosmos/tests/emulator_tests/mod.rs |
Registers the new cosmos_hpk module (correct alphabetical placement). |
sdk/cosmos/azure_data_cosmos/CHANGELOG.md |
Other Changes entry; correctly placed but lists null coverage that no test provides. |
Comment on lines
+136
to
+138
| /// All emulator query tests in this suite use the page iterator (`into_pages`) | ||
| /// rather than the item-level `TryStream`, because cross-partition fan-out is | ||
| /// only driven correctly through the paged interface. |
Member
Author
There was a problem hiding this comment.
Fixed in d97e667 — scoped the claim: only the cross-partition fan-out cases require into_pages; single-partition tests may use either interface.
The five gap tests carried both a #[cfg_attr(not(emulator), ignore)] gate and a plain #[ignore]. In non-emulator CI the cfg_attr expands to #[ignore], making the second attribute a duplicate that trips unused_attributes under -Dwarnings and fails the build on every non-Windows leg. Gap tests must be ignored unconditionally (they assert behavior the SDK lacks, so they'd also fail the live emulator leg), so drop the cfg_attr gate and keep a single #[ignore]; note how to re-enable. Also address review feedback on doc accuracy: - Drop the 'null' component-value claim (no null-component test exists) from the module doc and CHANGELOG. - Correct the query_geo_prefix doc to state prefix queries are NOT filtered today (was implying server-side filtering). - Scope the collect_query into_pages rationale to cross-partition only. - Retitle the feed-range negatives as HPK (not single-hash) negatives. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
simorenoh
changed the title
Closes emulator-reachable gaps found when reconciling PR #4682 against the Class A/B test matrix posted on issue #4155: - hpk_item_wrong_partition_read_not_found (a05): reading an existing id from a different valid full partition returns 404. - hpk_query_secondary_path_filter_servable (b03): a SQL filter on a non-leading partition-key path within a routed partition is served (server-side SQL filtering works even though prefix routing does not filter, per #4680). - hpk_query_single_partition_order_by_servable (b04): ORDER BY scoped to a single logical partition is servable. - hpk_query_cross_partition_advanced_not_servable (b06): DISTINCT, aggregate, GROUP BY, and OFFSET/LIMIT over full_container all fail with 400/1004 CrossPartitionQueryNotServable. - hpk_query_full_key_equality_cross_partition (b07): ignored; full-key equality over full_container panics the planner today (topology/EPK overlap) and needs PR #4638 (#4574) to become servable. The leading-path cross-partition filter case (b05) is already covered by the existing hpk_query_cross_partition_full_container (#4681). Live 2-partition split coverage (Class C) remains a follow-up requiring a provisioned account. Verified against the local emulator: 23 passed, 0 failed, 6 ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds an `in_memory_emulator_tests::cosmos_hpk_split` module covering hierarchical partition keys over a multi-physical-partition (split) topology, which the single-partition Cosmos emulator container used by `emulator_tests::cosmos_hpk` cannot reproduce. The suite drives the public `azure_data_cosmos` client against the in-memory emulator (gated on the internal `__internal_in_memory_emulator` feature, no live account required) and asserts: - topology: a 2-partition MultiHash container reports 2 feed ranges - full-key targeting: a 3-level key resolves to exactly one partition - distribution: a multi-country dataset spreads deterministically across both partitions via `feed_range_from_partition_key` routing, and a cross-partition query fans out and returns the whole dataset - split preservation: `split_partition` grows the topology to 2 ranges while data stays readable and correctly routed Documents two harness limitations: the in-memory query engine does not honor `FeedScope::range` filtering, and its geometric-midpoint split cannot produce deterministic cross-partition prefix fan-out (both validated against the real emulator elsewhere). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The b03 test hpk_query_secondary_path_filter_servable used a 2-level prefix scope (USA, CA) on a 3-level container. Prefix scoping lowers to a single-point EPK (the #4680 gap), which the CI Cosmos emulator rejects with 400 BadRequest even though the local emulator tolerated it. Rescope to the full 3-level key (USA, CA, SanFrancisco) while keeping the non-leading /city path filter, so the test still demonstrates the server serving a SQL predicate on a secondary PK path but stays robust across emulator versions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…-coverage-matrix # Conflicts: # sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_feed_ranges.rs
Now that #4729 (merged from main) fixes the HPK prefix (#4680) and cross-partition (#4681) query bugs, flip the ignores the PR deferred to this coverage branch: - hpk_query_cross_partition_full_container (#4681): enabled on both the classic and vnext emulators. Verified passing against the classic emulator locally. - hpk_query_prefix_level1/level2/no_match/correctness_guard (#4680): gated to run only on the vnext emulator (test_category=emulator_vnext). The classic Cosmos emulator rejects the prefix EPK-range query that #4729 emits with 400 BadRequest, so these stay ignored on the blocking classic legs. The prefix-filtering fix is already covered deterministically in CI by in_memory_emulator_tests::hpk. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the end-to-end Hierarchical Partition Key (HPK) test gap tracked in #4155 by adding a dedicated
emulator_tests::cosmos_hpkmodule and extendingcosmos_feed_rangeswith HPK negative-validation tests. All tests were authored and run against a live local Cosmos emulator (19 pass, 5 intentionally ignored, 0 fail).What's covered (passing)
400 / 1001 PartitionKeyMismatch.FeedScope::partition(full_key)).207 MultiStatusresponse with a per-op400(transaction does not commit).cosmos_feed_ranges.rs) — empty key →CLIENT_PARTITION_KEY_EMPTY, too-many components →CLIENT_PARTITION_KEY_TOO_MANY_COMPONENTS, and a multi-component key on a single-hash container →CLIENT_PARTITION_KEY_TOO_MANY_COMPONENTS.Intentionally ignored — verified product gaps
Five tests are written but
#[ignore]d because they surfaced real SDK behavior gaps (not test defects). They flip to positive once the underlying issues are fixed:FeedScope::partitiondocuments prefixes as unsupported; therange()route resolves whole physical partitions). Tracked in Cosmos: HPK prefix queries are not filtered (return all items) #4680 (related: PKRange resolution API follow-ups from PR #4342 #4380, Add Gateway 2.0 support to Cosmos Rust driver and SDK #4193).full_container()) queries over HPK containers fail with400 BadRequest("One of the input values is invalid"). The same shape works on single-hash containers. Tracked in Cosmos: cross-partition queries over HPK containers fail with 400 #4681.The C3 test also documents (in-code) that the
CLIENT_PREFIX_PARTITION_KEY_REQUIRES_MULTIHASHvalidation branch is unreachable for single-path containers.Notes
cargo fmtandcargo clippyare clean.Relates to #3666.