[OPIK-7110] [BE] fix: stop capping user-mapped variables on the agentic-tools path#7414
Open
miguelgrc wants to merge 2 commits into
Open
[OPIK-7110] [BE] fix: stop capping user-mapped variables on the agentic-tools path#7414miguelgrc wants to merge 2 commits into
miguelgrc wants to merge 2 commits into
Conversation
…ic-tools path On the agentic-tools path, capReplacements() was truncating ALL variable substitutions to 4K chars — including the user's scoring variables (output, expected_output). The LLM then had to drill down via tools to recover the full values, which was non-deterministic and produced intermittent "Insufficient data" failures. Now capReplacements() accepts an uncappedKeys set: user-mapped variables (trace sections and literals) are passed through at full length, while sentinel variables (spans, trace, span) are still capped since the tools are purpose-built to navigate them.
github-actions
Bot
added
Backend
tests
Contributor
⏱️ pre-commit per-hook timing
⏭️ 28 skipped (no matching files changed)
|
online_scoring_agentic_routing_total{path, trigger, workspace_id,
workspace_name} — emitted per evaluation in shouldUseAgenticTools,
tracks agentic vs inline routing by trigger reason and workspace.
Contributor
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
andrescrz
approved these changes
Jul 9, 2026
andrescrz
left a comment
andrescrz
left a comment
Member
There was a problem hiding this comment.
Couple of thoughts, but not blockers.
| * sentinel variables (like {@code spans}). Used by {@link #capReplacements} to decide which | ||
| * keys to leave uncapped. | ||
| */ | ||
| static Set<String> userMappedVariableKeys(Map<String, String> variables) { |
Member
There was a problem hiding this comment.
Not a blocker: I'd try to test this indirectly through the public methods, instead of unit testing, that should give you enough coverage in most of the cases. But if you still want to unit test this, please annotate it with VisibleForTesting from Guava.
Comment on lines
+75
to
+76
| private static final AttributeKey<String> PATH_KEY = AttributeKey.stringKey("path"); | ||
| private static final AttributeKey<String> TRIGGER_KEY = AttributeKey.stringKey("trigger"); |
Member
There was a problem hiding this comment.
These two metric are going to provide a lot of cardinality. Not a blocker, but you should keep an eye to see if this behaves well in production and check that it doesn't overwhelm our telemetry pipeline (collection, ingestion, dashboards etc.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Details
On the agentic-tools path,
capReplacements()was truncating ALL variable substitutions to 4K chars — including user-mapped scoring variables (output,expected_output). The LLM judge then had to drill down via tools to recover the full values, which was non-deterministic and produced intermittent "Insufficient data" failures for customers with large traces.capReplacements()now accepts anuncappedKeysset: user-mapped variables (trace sections and literals) pass through at full length, while sentinel variables (spans,trace,span) are still capped since the agentic tools are purpose-built to navigate them.prepareLlmRequestoverloads use the new selective capping.capReplacements(replacements, max, hint)is removed; callers passSet.of()for uniform capping.Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
Testing
mvn test -f apps/opik-backend/pom.xml -Dtest=\"OnlineScoringEngineCapReplacementsTest,OnlineScoringEngineTest\"— 108 tests pass, 0 failuresmvn spotless:check— passesrepro_opik_7110.py: created an LLM-as-judge rule withoutput/expected_outputvariable mappings, logged traces with 240K+ char output (answer buried past the 4K cap mark), confirmed agentic-tools path activates (Trace context exceeds '50000' tokens; switching to agentic-tools mode), and verified:answer_present=[0](LLM failed to drill down and find the answer)Documentation
N/A — internal behavioral fix, no API or config changes.