Repository navigation
Fix RubyLLM 2.x tool-calling turns and cache token counts - #243
Merged
Merged
Conversation
ruby_llm 2.x runs the whole tool loop inside Chat#complete (step until complete?), so wrapping complete gave one span per turn. On tool-calling turns that span recorded the tool-call request as its output and only the last request's tokens. On 2.x, trace Chat#generate instead, which step calls once per model request. 1.x keeps wrapping complete, which recurses after tool calls. The variant is picked when the patch is applied, by whether the class defines a public generate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tracing generate() alone made a tool-calling turn's chat and tool spans siblings, so a caller without its own span got one trace per request. On 2.x, complete() now opens a task span that parents them. It carries the turn's input and final output but no metrics, since each request's tokens are on its ruby_llm.chat span. 1.x is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Filtering to ruby_llm.chat alone let a double-wrapped complete() pass the idempotency test. Also expect exactly one ruby_llm.complete span on 2.x and none on 1.x. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruby_llm 2.x renamed the cache counts to cache_read_tokens and cache_write_tokens, so cached input never reached prompt_tokens and a cached request reported only its few uncached tokens. Read either name in one helper shared by the streaming and non-streaming paths. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
format_tool_schema deleted strict and additionalProperties from the tool's parameters hash in place unless it was frozen. That hash is the tool's cached schema, so instrumenting a chat stripped the constraints from the requests RubyLLM sends to the provider, for every later instance and even for untraced chats. Build trimmed copies instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
faraday-net_http 3.5.0 (2026-10-11) is past faraday 2.14.4's `< 3.5` cap, so the ruby_llm appraisals backtracked to faraday 2.0.0, which cannot load the net_http adapter, and every recorded RubyLLM test errored. Require faraday >= 2.14 there. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
David Elner (delner)
force-pushed
the
fix/ruby-llm-2-turn-spans
branch
from
November 16, 2026 16:32
0fd3842 to
44d16e5
Compare
Collaborator
Author
|
Ritchie C (@infinityrobot) this was released in 0.6.1. Thanks for your contribution! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A fork of #230 to fix CI, get this merged; thanks Ritchie C (@infinityrobot)
RubyLLM 2.x runs the whole tool loop inside
Chat#complete, so tracing that method alone drops the final answer from the chat span and records only the last request's tokens. This traces each model request throughChat#generate, groups a turn under aruby_llm.completespan, captures the 2.x cache-token fields, and avoids mutating tool schemas while recording metadata. The 1.x tracing path is preserved.Fixes #226.