mirror of
https://github.com/nesquena/hermes-webui.git
synced 2026-07-16 04:30:17 +00:00
nesquena-hermes
af0835a083
WebUI's api/streaming.py:_resolve_image_input_mode was a LOCAL re-implementation of agent/image_routing.py:decide_image_input_mode that had DIVERGED from the canonical function. The local copy never consulted the active model's vision capability — it only checked a handful of config heuristics — so it returned "native" (embedding the image as an image_url part) even for models KNOWN to lack vision. That silently sends pixels to a blind model (#21160-class failure) instead of routing through the text (vision_analyze) pipeline. Fix: delegate to the canonical agent.image_routing.decide_image_input_mode (single source of truth), with a try/except fallback that preserves the historical WebUI behaviour when the agent package is not importable (e.g. the WebUI standalone test environment). Carve-out preserved: the canonical router conservatively returns "text" for UNKNOWN/custom models (no models.dev capability data). WebUI historically forwards those NATIVELY and relies on the agent's strip-and-retry guard (_try_shrink_image_parts_in_messages / _strip_images_from_messages) to downgrade on a provider rejection. We keep that: a canonical "text" verdict is only honoured when there's a real signal — an explicit user choice (image_input_mode: text or a configured auxiliary.vision backend) or a model KNOWN to lack vision. Otherwise we forward native. Tests: - Existing gateway test (forwards_image_attachments_as_multimodal_parts) stays green — unknown "test-model" still forwards native. - New non-vacuous regression: a model KNOWN to be text-only (supports_vision == False) now routes "text" — this FAILS against master, where the old local copy returned "native" for the same config. - Added coverage for the unknown-model carve-out, known-vision native path, explicit-text-signal precedence, and the agent-unavailable fallback. Salvage of #4113 by @gkd2323c. The original PR delegated fully to canonical but did not add the unknown-model carve-out, which would have regressed the WebUI native-forwarding behaviour the gateway test encodes. Co-authored-by: gkd2323c <gkd2323c@users.noreply.github.com>