Add SemanticSearcher for semantic search#195
Open
abetomo wants to merge 3 commits into
Open
Conversation
It basically just adds language_model_knn() to the Searcher's filter. Overrode the keyword search only parts (e.g. `match_columns`) to do nothing.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new FullTextSearch::SemanticSearcher intended to run semantic (embedding/KNN-based) search via PGroonga’s language_model_knn(), and updates the existing select plumbing so callers can target a specific PGroonga index.
Changes:
- Add
SemanticSearcher(subclass ofSearcher) that builds a semantic KNN filter and targets a dedicated semantic index. - Extend the
Searcher/backendselectAPIs to allow passing an explicitindex_name. - Add a unit test for semantic search (guarded behind
SEMANTIC_SEARCH_TESTand PostgreSQL).
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
lib/full_text_search/semantic_searcher.rb |
Adds semantic-search-specific searcher that uses language_model_knn and a semantic index. |
test/unit/full_text_search/semantic_searcher_test.rb |
Adds an opt-in test that provisions the semantic index and asserts semantic ranking. |
lib/full_text_search/searcher.rb |
Allows subclasses to override index_name and slice usage; passes index_name into Target.select. |
lib/full_text_search/pgroonga.rb |
Updates select/build_sql to support choosing a PGroonga index name. |
lib/full_text_search/mroonga.rb |
Updates select signature to accept (and ignore) index_name for API compatibility. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+22
to
+25
| def knn_expression | ||
| query = Groonga::Client::ScriptSyntax.format_string(@request.query) | ||
| %Q[language_model_knn(content, #{query}, {"k": #{K}})] | ||
| end |
Contributor
Author
There was a problem hiding this comment.
In the initial implementation, K is fixed at 50.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
kou
reviewed
Jun 12, 2026
Contributor
Author
There was a problem hiding this comment.
This is how it's used.
--- a/lib/full_text_search/hooks/controller_search_index.rb
+++ b/lib/full_text_search/hooks/controller_search_index.rb
@@ -29,7 +29,12 @@ module FullTextSearch
return
end
- searcher = Searcher.new(@search_request)
+ if @search_request.semantic? && SemanticIndex.exist?
+ searcher = SemanticSearcher.new(@search_request)
+ else
+ searcher = Searcher.new(@search_request)
+ end
+
@result_set = searcher.search
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It basically just adds language_model_knn() to the Searcher's filter.
Overrode the keyword search only parts (e.g.
match_columns) to do nothing.