Add prefix and wildcard query support to DSL query executor#22525
Add prefix and wildcard query support to DSL query executor#22525ask-kamal-nayan wants to merge 8 commits into
Conversation
…ueries (cherry picked from commit 68b4363) Signed-off-by: Kamal Nayan <askkamal@amazon.com>
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
PR Reviewer Guide 🔍(Review updated until commit f165637)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to f165637 Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 84e30fc
Suggestions up to commit ac8a7be
Suggestions up to commit f3cffa8
Suggestions up to commit 16b0cde
|
|
❌ Gradle check result for 16b0cde: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
|
Persistent review updated to latest commit f3cffa8 |
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #22525 +/- ##
=========================================
Coverage 73.48% 73.48%
- Complexity 76460 76468 +8
=========================================
Files 6104 6104
Lines 346582 346582
Branches 49879 49879
=========================================
+ Hits 254675 254678 +3
+ Misses 71664 71624 -40
- Partials 20243 20280 +37 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Persistent review updated to latest commit ac8a7be |
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
|
Persistent review updated to latest commit 84e30fc |
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
|
❌ Gradle check result for 84e30fc: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Kamal Nayan <askkamal@amazon.com>
|
Persistent review updated to latest commit f165637 |
Description
Adds DSL-to-Calcite translation for
prefixandwildcardqueries in the sandboxdsl-query-executorplugin.This PR supersedes #21362 and carries forward @abhishek00159's original implementation rebased onto current main with a correctness fix for wildcard backslash-escape handling and hardened tests.
Summary
Implements
PrefixQueryTranslatorandWildcardQueryTranslatorthat convert OpenSearch prefix and wildcard queries to Calcite LIKE expressions with full support for case-insensitive matching and proper SQL special character escaping.Features
Prefix Query
Converts prefix queries to SQL LIKE with trailing wildcard:
{"prefix": {"name": "lap"}} // Converts to: name LIKE 'lap%' {"prefix": {"name": {"value": "LAP", "case_insensitive": true}}} // Converts to: LOWER(name) LIKE 'lap%'Supported parameters:
value— The prefix stringcase_insensitive— Case-insensitive matching (default: false)Unsupported parameters (throw ConversionException):
boost— Query boosting not supportedrewrite— Rewrite method not supportedWildcard Query
Converts wildcard patterns to SQL LIKE with pattern translation:
{"wildcard": {"name": "lap*"}} // Converts to: name LIKE 'lap%' {"wildcard": {"name": "l?ptop"}} // Converts to: name LIKE 'l_ptop' {"wildcard": {"name": {"value": "BOOK*", "case_insensitive": true}}} // Converts to: LOWER(name) LIKE 'book%'Wildcard characters:
*%?_\**\??\\\\\%/_in valueSupported parameters:
value— The wildcard patterncase_insensitive— Case-insensitive matching (default: false)Unsupported parameters (throw ConversionException):
boost— Query boosting not supportedrewrite— Rewrite method not supportedSpecial Character Escaping
Both translators properly escape SQL LIKE special characters in user data:
%→\%(prevents unintended SQL wildcard)_→\_(prevents unintended SQL single-char match)\→\\(escape character itself)Example:
{"prefix": {"path": "C:\\test_"}} // Converts to: path LIKE 'C:\\test\_%' {"wildcard": {"name": "a\\*b_c*"}} // Converts to: name LIKE 'a*b\_c%'Case-Insensitive Support
When
case_insensitive: true:LOWER()function to the field referenceWildcard Escape Fix (new in this PR)
The original implementation escaped
\without lookahead, so\*became\\%— an escaped backslash followed by a match-anything wildcard — instead of a literal*. The converter is now a single-pass state machine matchingWildcardQueryBuildersemantics correctly. This was verified TDD-first: 6 red tests confirmed the bug before the fix, then went green after.Examples
Prefix in Bool Query:
{ "bool": { "must": [{"prefix": {"category": "electron"}}], "should": [ {"prefix": {"brand": "sam"}}, {"prefix": {"brand": "app"}} ] } }Converts to:
(category LIKE 'electron%') AND ((brand LIKE 'sam%') OR (brand LIKE 'app%'))Complex Wildcard Pattern:
{"wildcard": {"sku": "*-2021-*"}}Converts to:
sku LIKE '%-2021-%'Testing
Unit Tests:
\*,\?,\\), mixed patterns, error handlingIntegration Tests:
DslPrefixQueryIT— end-to-end validation with real index dataDslWildcardQueryIT— end-to-end validation with real index data@AwaitsFixpending analytics engine E2E pipeline wiring (consistent with all sibling ITs in the module)Manual testing (from original PR):
Prefix:
Wildcard:
Known Gaps (deliberate, consistent with sibling translators)
boostandrewriteare rejected rather than emulatedRelated Issues
Check List
--signoffBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.