Improve feedback-driven recall ranking#70
Conversation
📝 WalkthroughWalkthroughThis PR adds identifier-aware recall data and ranking, Neo4j v2 recall indexes with backfills, feedback recall evaluation seeds, fragment evidence fallback handling, and related-dream filtering and reranking. ChangesIdentifier-aware recall platform
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant RecallMemory
participant RecallService
participant Neo4j
participant DreamReranker
RecallMemory->>RecallService: recall query
RecallService->>Neo4j: retrieve recall hits
RecallMemory->>Neo4j: overfetch related dreams
Neo4j-->>RecallMemory: candidate dreams
RecallMemory->>DreamReranker: rerankRelatedDreams(query, hits, dreams)
DreamReranker-->>RecallMemory: filtered and capped related dreams
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
gitvibe-for-github
Bot
added
the
gvi:reviewing
![gitvibe-for-github[bot]](https://avatars.githubusercontent.com/in/4002928?s=60&v=4){kind=small}
There was a problem hiding this comment.
GitVibe Risk Accepted
Status: blocked
Risk: accepted for matching context
Z-M-Huang accepted this prompt-injection input risk for matching review-matrix context.
GitVibe replaced the previous blocked result details to keep this thread readable.
Accepted Risk
Z-M-Huang accepted this prompt-injection input risk for matching context.
Accepted at: 2026-07-10T00:28:27.775Z
Accepted workflow run: 29059688967
Accepted workflow attempt: 2
Accepted stages: review-matrix
Artifact title/body SHA: 27d9169c051a0968ee5a27450941cf529f48dba27a8216e5ced5bbf3f73f4042
Pull request head SHA: 9a46c0fabb278ea2d63b8653ff3faa814d2b0e60
gitvibe-for-github
Bot
added
gvi:blocked
Z-M-Huang
added
the
git-vibe:accept-risk
gitvibe-for-github
Bot
added
gvi:reviewing
There was a problem hiding this comment.
GitVibe Risk Accepted
Status: blocked
Risk: accepted for matching context
Z-M-Huang accepted this prompt-injection input risk for matching review-matrix context.
GitVibe replaced the previous blocked result details to keep this thread readable.
Accepted Risk
Z-M-Huang accepted this prompt-injection input risk for matching context.
Accepted at: 2026-07-10T01:26:29.973Z
Accepted workflow run: 29059688967
Accepted workflow attempt: 3
Accepted stages: review-matrix
Artifact title/body SHA: 27d9169c051a0968ee5a27450941cf529f48dba27a8216e5ced5bbf3f73f4042
Pull request head SHA: 9a46c0fabb278ea2d63b8653ff3faa814d2b0e60
gitvibe-for-github
Bot
added
gvi:blocked
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9a46c0fabb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if len(queryHardAnchors) > 0 && !recallident.SatisfiesAnchors(queryHardAnchors, m.Content, m.RecallText) { | ||
| continue |
There was a problem hiding this comment.
When a query contains hard anchors but none of the overfetched fragment candidates contains every anchor, this branch drops every fragment candidate. For example, a low-limit query that mentions both a PR and a commit can retrieve only partial/legacy matches in the branch pool; instead of returning the best RRF-ranked fallback, recall returns only tier hits or an empty fragment list. Please first check whether any candidate satisfies the hard anchors, and only apply this filter when that set is non-empty.
Useful? React with 👍 / 👎.
| "fact_predicate_idx", | ||
| }, | ||
| { | ||
| "CREATE FULLTEXT INDEX fragment_recall_v2_idx IF NOT EXISTS FOR (sf:SourceFragment) ON EACH [sf.content, sf.source, sf.idempotency_key, sf.recall_text]", |
There was a problem hiding this comment.
The keyword branch now queries this v2 index, but it still omits metadata_json. Newly direct-imported eval corpus rows are written by internal/evalharness/direct_import.go without recall_text, and several new hardcases put required anchors only in CorpusItem.Metadata (for example PR/commit/tool identifiers), so those rows never expose the anchor terms to the index until an external backfill happens. This makes the new metadata hardcases miss the required fragment and favor decoys whose body text contains the terms; either index metadata_json here or populate recall_text during direct import.
Useful? React with 👍 / 👎.
| if strings.Contains(token, "/") || strings.Contains(token, ":") || strings.Contains(token, "_") { | ||
| return true |
There was a problem hiding this comment.
Typed fact/claim recall calls SatisfiesStrongAnchors, so this rule makes any slash/colon/underscore token an exact gate even when it is ordinary prose. For queries such as TCR/CD3 or U.S. economy, full-text search can retrieve a relevant fact whose text normalizes the punctuation differently (TCR and CD3, US economy), but the post-filter drops it because Extract cannot recreate the exact slash/dotted token. This broad-token gating was avoided for fragment hard anchors, but facts and claims still lose otherwise valid recall results.
Useful? React with 👍 / 👎.
| recall_text: $recallText, | ||
| identifier_tokens: $identifierTokens |
There was a problem hiding this comment.
This only initializes recall fields when a new Fact is created. When handleSingleCurrent reuses an existing same-object fact via linkClaimToExistingFact, the confirming claim is marked superseded but its recall_text and identifier_tokens are never merged into the existing Fact; if that claim carried a new metadata/source identifier, the claim is no longer eligible for validated-claim recall and the Fact cannot be found by the new anchor. Please update the existing Fact's recall fields in the reuse path as well.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (4)
internal/tools/registry/recall_dream_filter_test.go (1)
30-43: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd order assertions to verify reranking behavior.
The test verifies which dreams are included/excluded but doesn't assert their order. For a reranking function, the order is a key behavior — a wrong order with the right set would pass the current test but produce incorrect results in production.
✨ Proposed addition
for _, want := range []string{"dream-related-1", "dream-related-2", "dream-related-3"} { if !gotIDs[want] { t.Fatalf("related dreams missing %s: %+v", want, gotIDs) } } for _, notWant := range []string{"dream-related-4", "dream-unrelated", "dream-stale"} { if gotIDs[notWant] { t.Fatalf("related dreams included %s: %+v", notWant, gotIDs) } } + wantOrder := []string{"dream-related-1", "dream-related-2", "dream-related-3"} + for i, wantID := range wantOrder { + if got[i].DreamID != wantID { + t.Fatalf("related dreams[%d] = %s; want %s", i, got[i].DreamID, wantID) + } + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/tools/registry/recall_dream_filter_test.go` around lines 30 - 43, Add assertions in the recall/reranking test after collecting got so it verifies the returned dreams appear in the expected reranked order, not merely the expected set. Compare each result’s DreamID against the intended ordered sequence and fail with the actual order when mismatched, while retaining the existing inclusion/exclusion checks.internal/service/claimservice/create.go (1)
434-470: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winNon-deterministic
parts/token ordering from map iteration.
for key, value := range classification(Line 447) andfor key, value := range fragment.Metadata(Line 462) iterate over Go maps, whose order is randomized per run. This meansrecallText/identifierTokensgenerated for logically identical claims can differ across create calls or backfill re-runs, undermining the deterministic-recall goal called out in the PR objectives. Functionally the deduped token set is likely unaffected (via theMergeTokenscollector), but the persistedrecall_textstring is not reproducible, which will also make any exact-string assertions in tests or backfill diffing flaky.♻️ Proposed fix: sort map keys before iterating
+ classKeys := make([]string, 0, len(classification)) + for key := range classification { + classKeys = append(classKeys, key) + } + sort.Strings(classKeys) + for _, key := range classKeys { + parts = append(parts, key, fmt.Sprint(classification[key])) + } - for key, value := range classification { - parts = append(parts, key, fmt.Sprint(value)) - } tokenGroups := make([][]string, 0, len(fragments)+1) for _, fragment := range fragments { if fragment == nil { continue } parts = append(parts, fragment.Content, fragment.Source, fragment.IdempotencyKey, fragment.RecallText, ) parts = append(parts, fragment.Labels...) - for key, value := range fragment.Metadata { - parts = append(parts, key, fmt.Sprint(value)) - } + metaKeys := make([]string, 0, len(fragment.Metadata)) + for key := range fragment.Metadata { + metaKeys = append(metaKeys, key) + } + sort.Strings(metaKeys) + for _, key := range metaKeys { + parts = append(parts, key, fmt.Sprint(fragment.Metadata[key])) + } tokenGroups = append(tokenGroups, fragment.IdentifierTokens) }Requires adding
"sort"to imports.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/service/claimservice/create.go` around lines 434 - 470, Make claimRecallFields deterministic by sorting map keys before appending classification and fragment metadata entries. Add the sort import, collect and sort keys for classification and each fragment.Metadata map, then iterate in sorted order so recallText and identifierTokens are reproducible.internal/service/dreamservice/service.go (1)
938-950: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCross-file duplicate of the
[]any→[]stringfilter helper.
stringSliceFromMaphere duplicates the same filtering logic asstringSliceininternal/service/factservice/promote_rows.go(lines 51-63). Both were introduced in this PR for the same identifier-tokens decoding need.Consider extracting a single shared helper (e.g. in a small internal util package) used by both
dreamserviceandfactservice(and any other callers) instead of maintaining parallel implementations.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/service/dreamservice/service.go` around lines 938 - 950, Extract the duplicated []any-to-[]string filtering logic from stringSliceFromMap and factservice’s stringSlice into one shared internal utility helper, then update both callers to use it while preserving empty-string filtering and invalid-type handling.internal/service/factservice/promote_rows.go (1)
51-73: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReuse the new
stringSlicehelper forsupportedByinstead of duplicating the same logic.The new
stringSlicehelper (lines 51-63) and the inlinesupportedByparsing (lines 65-73) are functionally identical.♻️ Proposed fix
- var supportedBy []string - if raw, ok := row["supported_by"].([]any); ok { - supportedBy = make([]string, 0, len(raw)) - for _, v := range raw { - if s, ok := v.(string); ok && s != "" { - supportedBy = append(supportedBy, s) - } - } - } + supportedBy := stringSlice("supported_by")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/service/factservice/promote_rows.go` around lines 51 - 73, Reuse the existing stringSlice helper when assigning supportedBy, replacing the duplicated type assertion and filtering loop with a call using the "supported_by" key.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/storage/neo4j/schema.go`:
- Around line 560-568: Ensure backfillRecallSearchText is not run
unconditionally on every EnsureSchema invocation. Add an idempotent one-time
migration marker, such as a SchemaMigration node keyed by migration name, and
gate the four label backfill queries behind an atomic check/create so completed
backfills are skipped on subsequent startups; preserve retry behavior if the
backfill fails.
- Around line 573-585: Update the schema migration entry identified by
“sourcefragment_recall_text” and the corresponding Claim, Fact, and Dream
backfills to recompute recall_text and identifier_tokens through the shared
recallident builders used by write paths, rather than inline Cypher and empty
arrays. Implement a real migration that loads legacy records, applies the common
helper, and persists both derived fields consistently with new records.
In `@internal/tools/registry/recall_dream_filter.go`:
- Around line 34-35: Update the identifierOverlap calculation in the recall
filter to exclude the metadata fields dream.CycleRunID and dream.GeneratorModel
from recallident.OverlapText, while retaining the substantive dream content
fields.
---
Nitpick comments:
In `@internal/service/claimservice/create.go`:
- Around line 434-470: Make claimRecallFields deterministic by sorting map keys
before appending classification and fragment metadata entries. Add the sort
import, collect and sort keys for classification and each fragment.Metadata map,
then iterate in sorted order so recallText and identifierTokens are
reproducible.
In `@internal/service/dreamservice/service.go`:
- Around line 938-950: Extract the duplicated []any-to-[]string filtering logic
from stringSliceFromMap and factservice’s stringSlice into one shared internal
utility helper, then update both callers to use it while preserving empty-string
filtering and invalid-type handling.
In `@internal/service/factservice/promote_rows.go`:
- Around line 51-73: Reuse the existing stringSlice helper when assigning
supportedBy, replacing the duplicated type assertion and filtering loop with a
call using the "supported_by" key.
In `@internal/tools/registry/recall_dream_filter_test.go`:
- Around line 30-43: Add assertions in the recall/reranking test after
collecting got so it verifies the returned dreams appear in the expected
reranked order, not merely the expected set. Compare each result’s DreamID
against the intended ordered sequence and fail with the actual order when
mismatched, while retaining the existing inclusion/exclusion checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: f2bbbeb2-de54-4ffd-bec1-87f74090d985
📒 Files selected for processing (42)
cmd/eval-seedgen/feedback_recall_hardcases.gocmd/eval-seedgen/main.gointernal/domain/claim.gointernal/domain/dream.gointernal/domain/fact.gointernal/domain/fragment.gointernal/evalharness/runner_evidence_fallback_test.gointernal/evalharness/runner_test.gointernal/evalharness/score.gointernal/recallident/identifier.gointernal/recallident/identifier_test.gointernal/service/claimservice/create.gointernal/service/claimservice/read.gointernal/service/claimservice/support.gointernal/service/dreamservice/recall_fields.gointernal/service/dreamservice/service.gointernal/service/dreamservice/service_test.gointernal/service/factservice/promote.gointernal/service/factservice/promote_rows.gointernal/service/factservice/read.gointernal/service/fragmentservice/create.gointernal/service/fragmentservice/list.gointernal/service/fragmentservice/read.gointernal/service/recallservice/recall.gointernal/service/recallservice/recall_fragment_limit_test.gointernal/service/recallservice/recall_identifier.gointernal/service/recallservice/recall_limit_test.gointernal/service/recallservice/recall_rerank.gointernal/service/recallservice/recall_rerank_test.gointernal/service/recallservice/recall_rrf.gointernal/service/recallservice/recall_temporal.gointernal/service/recallservice/recall_temporal_test.gointernal/service/recallservice/recall_test.gointernal/service/recallservice/searcher.gointernal/storage/neo4j/schema.gointernal/tools/keywordsearch/searcher.gointernal/tools/keywordsearch/service.gointernal/tools/registry/dream_tools_test.gointernal/tools/registry/evaluation_tools.gointernal/tools/registry/recall_dream_filter.gointernal/tools/registry/recall_dream_filter_test.gointernal/tools/registry/toolset.go
| if err := s.backfillRecallSearchText(ctx); err != nil { | ||
| return err | ||
| } | ||
|
|
||
| s.logger.Info("Neo4j schema ensured successfully") | ||
| return nil | ||
| } | ||
|
|
||
| func (s *SchemaBootstrapper) backfillRecallSearchText(ctx context.Context) error { |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟠 Major | ⚡ Quick win
Unconditional full-label scans on every EnsureSchema call.
Unlike the surrounding IF NOT EXISTS index/constraint statements, these four backfill queries always MATCH the entire label (SourceFragment/Claim/Fact/Dream) to evaluate WHERE ... IS NULL, with no marker to skip once already backfilled. If EnsureSchema runs on every service startup/deploy (as its idempotent "ensure" pattern suggests), this adds a full scan of potentially large node sets on every boot.
Consider gating this behind a one-time marker (e.g. a SchemaMigration{name} node) so it only executes once rather than on every bootstrap.
Also applies to: 640-651
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/storage/neo4j/schema.go` around lines 560 - 568, Ensure
backfillRecallSearchText is not run unconditionally on every EnsureSchema
invocation. Add an idempotent one-time migration marker, such as a
SchemaMigration node keyed by migration name, and gate the four label backfill
queries behind an atomic check/create so completed backfills are skipped on
subsequent startups; preserve retry behavior if the backfill fails.
| { | ||
| name: "sourcefragment_recall_text", | ||
| cypher: ` | ||
| MATCH (sf:SourceFragment) | ||
| WHERE sf.recall_text IS NULL OR sf.identifier_tokens IS NULL | ||
| SET sf.recall_text = coalesce(sf.recall_text, trim( | ||
| coalesce(sf.content, '') + ' ' + | ||
| coalesce(sf.source, '') + ' ' + | ||
| coalesce(sf.idempotency_key, '') + ' ' + | ||
| reduce(acc = '', label IN coalesce(sf.labels, []) | acc + ' ' + toString(label)) + ' ' + | ||
| coalesce(sf.metadata_json, '') | ||
| )), | ||
| sf.identifier_tokens = coalesce(sf.identifier_tokens, [])`, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Look for any other identifier_tokens backfill/reindex mechanism.
rg -nP 'identifier_tokens' --type=go -g '!**/*_test.go' -C2Repository: markhuangai/dense-mem
Length of output: 159
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the schema file around the cited backfill sections and search for related backfill/reindex logic.
git ls-files | sed -n '1,200p'
printf '\n--- identifier_tokens references ---\n'
rg -n "identifier_tokens|recallident|backfillRecallSearchText|recall_text" internal . -g '!**/*_test.go' -g '!**/*.md' -g '!**/vendor/**' -C 2Repository: markhuangai/dense-mem
Length of output: 50377
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the cited area in internal/storage/neo4j/schema.go with line numbers.
wc -l internal/storage/neo4j/schema.go
sed -n '520,680p' internal/storage/neo4j/schema.go | cat -nRepository: markhuangai/dense-mem
Length of output: 6448
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- backfill / reindex references ---\n'
rg -n "backfill|reindex|recompute|refresh.*identifier|identifier_tokens" internal cmd -g '!**/*_test.go' -C 2
printf '\n--- recallident implementation ---\n'
sed -n '1,220p' internal/recallident/identifier.go | cat -nRepository: markhuangai/dense-mem
Length of output: 250
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- schema backfill line ranges ---\n'
sed -n '568,640p' internal/storage/neo4j/schema.go | cat -n
printf '\n--- recall text builders used on writes ---\n'
sed -n '210,260p' internal/service/fragmentservice/create.go | cat -n
sed -n '450,475p' internal/service/claimservice/create.go | cat -n
sed -n '585,605p' internal/service/factservice/promote.go | cat -n
sed -n '1,40p' internal/service/dreamservice/recall_fields.go | cat -nRepository: markhuangai/dense-mem
Length of output: 250
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- backfill / reindex references ---'
rg -n "backfill|reindex|recompute|refresh.*identifier|identifier_tokens" internal cmd -g '!**/*_test.go' -C 2 || true
echo
echo '--- recallident implementation ---'
sed -n '1,220p' internal/recallident/identifier.go | cat -nRepository: markhuangai/dense-mem
Length of output: 32263
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- schema backfill line ranges ---'
sed -n '568,640p' internal/storage/neo4j/schema.go | cat -n
echo
echo '--- recall text builders used on writes ---'
sed -n '220,240p' internal/service/fragmentservice/create.go | cat -n
sed -n '460,470p' internal/service/claimservice/create.go | cat -n
sed -n '590,600p' internal/service/factservice/promote.go | cat -n
sed -n '1,30p' internal/service/dreamservice/recall_fields.go | cat -nRepository: markhuangai/dense-mem
Length of output: 5755
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- exact identifier_tokens usages outside schema backfill ---'
rg -n "identifier_tokens\s*=" internal cmd -g '!**/*_test.go' -C 1 || trueRepository: markhuangai/dense-mem
Length of output: 1504
Backfill legacy recall fields with the shared builders
internal/storage/neo4j/schema.go:568-637 backfills legacy rows with identifier_tokens = [] instead of extracted tokens, and rebuilds recall_text inline rather than through the shared recallident path used on writes. That leaves pre-existing SourceFragment, Claim, Fact, and Dream rows with permanently weaker search/ranking metadata. Consider a real migration that recomputes both fields from the same helper used for new records.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/storage/neo4j/schema.go` around lines 573 - 585, Update the schema
migration entry identified by “sourcefragment_recall_text” and the corresponding
Claim, Fact, and Dream backfills to recompute recall_text and identifier_tokens
through the shared recallident builders used by write paths, rather than inline
Cypher and empty arrays. Implement a real migration that loads legacy records,
applies the common helper, and persists both derived fields consistently with
new records.
| identifierOverlap := recallident.Overlap(queryIDs, dream.IdentifierTokens) + | ||
| recallident.OverlapText(queryIDs, dream.Hypothesis, dream.WhatIf, dream.PossibleOutcome, dream.Rationale, dream.RecallText, dream.CycleRunID, dream.GeneratorModel) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Consider excluding metadata fields from identifier overlap.
dream.CycleRunID and dream.GeneratorModel are metadata, not content. Including them in recallident.OverlapText means a query mentioning a model name (e.g., "gpt-4") would boost all dreams generated by that model regardless of topical relevance, potentially degrading ranking quality.
🔧 Proposed fix
identifierOverlap := recallident.Overlap(queryIDs, dream.IdentifierTokens) +
- recallident.OverlapText(queryIDs, dream.Hypothesis, dream.WhatIf, dream.PossibleOutcome, dream.Rationale, dream.RecallText, dream.CycleRunID, dream.GeneratorModel)
+ recallident.OverlapText(queryIDs, dream.Hypothesis, dream.WhatIf, dream.PossibleOutcome, dream.Rationale, dream.RecallText)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| identifierOverlap := recallident.Overlap(queryIDs, dream.IdentifierTokens) + | |
| recallident.OverlapText(queryIDs, dream.Hypothesis, dream.WhatIf, dream.PossibleOutcome, dream.Rationale, dream.RecallText, dream.CycleRunID, dream.GeneratorModel) | |
| identifierOverlap := recallident.Overlap(queryIDs, dream.IdentifierTokens) + | |
| recallident.OverlapText(queryIDs, dream.Hypothesis, dream.WhatIf, dream.PossibleOutcome, dream.Rationale, dream.RecallText) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/tools/registry/recall_dream_filter.go` around lines 34 - 35, Update
the identifierOverlap calculation in the recall filter to exclude the metadata
fields dream.CycleRunID and dream.GeneratorModel from recallident.OverlapText,
while retaining the substantive dream content fields.
There was a problem hiding this comment.
GitVibe Risk Accepted
Status: blocked
Risk: accepted for matching context
Z-M-Huang accepted this prompt-injection input risk for matching review-matrix context.
GitVibe replaced the previous blocked result details to keep this thread readable.
Accepted Risk
Z-M-Huang accepted this prompt-injection input risk for matching context.
Accepted at: 2026-07-10T02:34:41.508Z
Accepted workflow run: 29059688967
Accepted workflow attempt: 4
Accepted stages: review-matrix
Artifact title/body SHA: 27d9169c051a0968ee5a27450941cf529f48dba27a8216e5ced5bbf3f73f4042
Pull request head SHA: 9a46c0fabb278ea2d63b8653ff3faa814d2b0e60
gitvibe-for-github
Bot
added
gvi:blocked
Z-M-Huang
added
the
git-vibe:accept-risk
gitvibe-for-github
Bot
added
gvi:reviewing
There was a problem hiding this comment.
GitVibe Review Matrix
Status: blocked
Next state: blocked
GitVibe paused this run for maintainer review.
Questions
- AI safety gate failed closed.
A. Rerun after the safety classifier runtime is healthy, or fix the safety configuration before rerunning.
Next Action
Reply in one comment with question numbers and option letters, or write your own answer for any question.
Result
When present, required fixes are posted as pull request review comments. Full structured output remains in the workflow run summary.
Workflow run: https://github.com/markhuangai/dense-mem/actions/runs/29059688967
Inline comments: 0
Required Fixes
- safety gate: AI safety gate failed: Your access token could not be refreshed because your refresh token was already used. Please log out and sign in again. - The AI safety classifier failed while the prompt-injection gate was enabled.
gitvibe-for-github
Bot
added
gvi:blocked
1 participant
What
Why
The feedback-driven candidate regressed because identifier metadata was not consistently searchable and broad terms could be promoted as exact project identifiers. The earlier fix recovered most of the loss, but still remained below the clean baseline. This change targets those failure modes without adding an LLM call to the online recall path.
Evaluation
Two complete 2,727-case runs were compared with the clean baseline:
The result is practical parity with small gains in Recall@K, MRR, and required rank-1; it is not a uniform win because nDCG remains 0.000116 below the clean baseline. Both runs scored all 2,727 cases with zero unmapped source references.
The targeted six-case hardcase suite reached 1.0 for context Recall@K, MRR, nDCG, and required rank-1. Known limits remain: source-type projection can make typed judgments look better than user-visible fragments, and the bad related dream is demoted rather than fully excluded.
Artifacts:
tests/eval/runs/public_rag_3axis_5k_v1_after_ranking_anchors_v1/summary.jsontests/eval/runs/public_rag_3axis_5k_v1_after_ranking_anchors_v1_repeat_01/summary.jsontests/eval/runs/feedback_recall_hardcases_v1_after_ranking_anchors_v1/summary.jsontests/eval/runs/compare_public_rag_3axis_5k_clean_vs_ranking_anchors_v1/comparison.jsontests/eval/runs/compare_public_rag_3axis_5k_clean_vs_ranking_anchors_v1_repeat_01/comparison.jsonRisk
Checks
go test ./cmd/... ./internal/...git diff --checkFollow-up
AI-assisted anchor extraction is intentionally not included here. The next proposal can compare an exact-span AI classifier as a soft ranking signal, first in offline/shadow evaluation, while deterministic parsing remains the hard safety boundary.
Summary by CodeRabbit
New Features
Bug Fixes