Repository navigation
test(escalation): thinking off for Gemma 4 as well as Qwen, through both backends - #43
Merged
Merged
Conversation
…oth backends The step-2b smoke run showed gemma4:e2b answering in ~30 visible tokens while spending up to 478 output tokens on hidden reasoning, against the 512-token cap. Qwen thinking was already off (#38), so gemma4 alone reasoned before answering: not equal terms, and one long reply from a length truncation. thinks_by_default() names Qwen and Gemma 4; build_chat_payload gives both think: false (the /no_think suffix stays Qwen-only; models that do not think still get no think field, since some refuse it). The socket backend forwards the payload's think field to the delegate when present. Persona: willow Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vcntd3qgAxd6gnPEvVukVm
rudi193-cmd
approved these changes
Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bite
Every model that thinks by default (Qwen, Gemma 4) answers with thinking off, through both backends, so all eight ladder models answer on the same terms.
Decision
Turn Gemma 4's thinking off rather than raise the output cap for everyone. Operator: "turn thinking off for gemma, and re run". Equal terms across the ladder was the design since #38 (Qwen thinking off); Gemma 4 was the one model still reasoning first.
What was done
thinks_by_default(model): Qwen (substringqwen) or Gemma 4 (prefixgemma4, case-insensitive).build_chat_payload:think: falsefor both; the/no_thinksuffix stays Qwen-only; models that do not think still get nothinkfield (some refuse it).socket_backend.chat_requestforwardsthinkto the delegate only when the payload carries it.Evidence
tokens_outup to 478 with ~30-token visible answers, alldone_reason: stop; every other model ≤ 73.python3 -m pytest -q tests/test_escalation_socket_backend.py tests/test_escalation_runner.py tests/test_escalation_fixtures.py→151 passed in 2.29s; privacy gate OK; ruff clean.thinks_by_default→4 failed; socket forward dropped →3 failed.Out of scope
0addc7f9) are unchanged.thinkthrough ships in feat(deterministic): chat op takes the caller's think flag willow-memory/willow-bot#84.Next bite
Re-run the gemma4 smoke to confirm output tokens drop, then the full 8-model ladder with
--resume.Ratified-by: sean-campbell — "turn thinking off for gemma, and re run"
🤖 Generated with Claude Code
https://claude.ai/code/session_01Vcntd3qgAxd6gnPEvVukVm