opencode-vision

An opencode plugin that automatically describes images attached to your messages, so models without native vision support can still "see" them.

How it works

When an image is attached to a message, the plugin:

1. Checks whether the current session's model supports images natively. If it does, nothing happens.

2. Otherwise calls a vision-capable model and injects the description into the conversation as:

   [opencode-vision] Image: <description>
   (saved: <path to the image on disk>)
   

3. Caches descriptions in memory (deduped by image hash), so re-attaching the same image never re-analyzes it.

The description is generated by the vision model in the same language as your message, so no UI text needs to be localized.

flowchart TD
    A["User attaches an image and submits"] --> B["transform hook runs on the message"]
    B --> C{"Session model has native vision?"}
    C -- "yes" --> D["No change — the model sees the image directly"]
    C -- "no" --> E["Strip previous [opencode-vision] hints from the message"]
    E --> F{"Any image parts?"}
    F -- "no" --> G["No change"]
    F -- "yes" --> H{"Mode?"}
    H -- "delegate" --> I["Save image to temp path"]
    I --> J["Replace part with a hint, instruct agent"]
    J --> K["Agent delegates to a vision subagent"]
    H -- "replace" --> L["For each image:"]
    L --> M["Save image & compute hash"]
    M --> N{"Description cached?"}
    N -- "yes" --> O["Reuse cached description"]
    N -- "no" --> P["Resolve vision backend"]
    P --> Q["Call vision API with a context-aware prompt"]
    Q --> R["Cache the description"]
    R --> S["Replace image part with the description text"]
    O --> S
    S --> T["Model sees the image content in the message flow"]

Loading

Modes

• replace (default): the image part is replaced inline with a text description from a vision model (OpenAI-compatible backends only). If no vision backend can be resolved, the plugin automatically falls back to delegate mode.
• delegate (auto): the image is saved to a temp path and the agent is instructed to delegate analysis to a vision subagent (e.g. @vision). The plugin falls back to delegate mode whenever it cannot resolve a replace-mode backend: no logged-in/configured vision-capable provider, or the chosen model isn't served over the OpenAI-compatible chat/completions protocol (see "Zen multi-protocol routing" below). On first use the plugin auto-creates ~/.config/opencode/agent/vision.md (a free opencode/mimo-v2.5-free vision subagent) if none exists — restart opencode once after creation. The delegate path uses opencode's own model routing (correct for every model family), so it never needs hand-crafted requests or endpoint whitelists. There is no config flag to force delegate mode; it is driven by backend availability and model protocol.

Install

From npm (recommended)

Add the package to the plugin array in opencode.json and restart opencode (it installs automatically at startup):

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@ariga39/opencode-vision"]
}

The package is self-contained (bundled), so no extra dependencies are needed. For a step-by-step, LLM-friendly walkthrough see INSTALL.md.

From source (manual)

Copy opencode-vision.ts into your opencode plugins directory and restart opencode.

# Linux / macOS
cp opencode-vision.ts ~/.config/opencode/plugins/

# Windows
copy opencode-vision.ts %USERPROFILE%\.config\opencode\plugins\

Configuration

Choosing the vision model

The plugin resolves the vision backend in this order:

1. A custom provider["vision-aux"] entry in opencode.json.
2. The choice file at ~/.config/opencode/vision-model.txt, containing a provider/model id.
3. The default opencode/mimo-v2.5-free.

You can also switch models from inside a session: ask the agent to run the vision_models tool to list candidates, then vision_set_model to persist a choice.

Free default: the default opencode/mimo-v2.5-free runs on the free tier of the zen gateway — you only need a free login key: opencode auth login. Other free zen models exist too (e.g. opencode/deepseek-v4-flash-free, opencode/hy3-free). The plugin briefly mentions this on first use and lets you switch anytime.

Zen multi-protocol routing: the zen gateway serves different model families over different APIs (OpenAI responses, Anthropic messages, Google, OpenAI-compatible chat/completions). Replace mode calls the OpenAI-compatible chat/completions API, so it only advertises models from compatible families (deepseek, minimax, glm, kimi, mimo, hy3, laguna, nemotron, big-pickle). If you choose a model from another family (e.g. opencode/qwen3.6-plus), the plugin automatically falls back to delegate mode, where the agent delegates to a vision subagent and opencode's own model routing handles the protocol — no hand-crafted request needed.

opencode.json

{
  "provider": {
    "vision-aux": {
      "options": {
        "baseURL": "https://example.com/v1",
        "apiKey": "sk-...",
        "model": "some-model"
      }
    }
  }
}

The provider API key is read from ~/.local/share/opencode/auth.json; config-declared providers are also discovered automatically.

Note: opencode validates its config against a schema and strips unknown keys, so plugin-specific keys such as experimental.vision.* are not forwarded to plugins. Choose the vision model with the vision-model.txt choice file or the vision_models / vision_set_model tools instead.

Limitations

• Replace mode speaks OpenAI-compatible APIs only. It hand-crafts chat/completions requests, so it works with OpenAI-compatible gateways and the zen families listed above. Models served via OpenAI responses, Anthropic messages, or Google protocols (e.g. qwen/claude/gpt/gemini on zen) are excluded and automatically fall back to delegate mode.
• Model catalog drift. Backend resolution reads the models.dev catalog (models.json), which updates frequently. A model can be listed in the catalog but not (yet) served by the API (e.g. kimi-k2.5-free), which surfaces as an injected error rather than a graceful fallback.
• Delegate needs a vision subagent. opencode ships no built-in vision agent. The plugin auto-creates ~/.config/opencode/agent/vision.md on first run, but a restart is needed for it to take effect; until then delegation may fail.
• Delegate is slower and not inline. It adds a subagent round-trip, and the description arrives as the subagent's output rather than inline in the message.
• Privacy. In replace mode the image is sent (as base64) to the configured vision backend; in delegate mode the subagent reads the saved image. Free zen models (e.g. mimo-v2.5-free) may collect data during their free period — see opencode's Zen docs.
• Native-vision detection relies on the session model's reported capabilities; a model that actually supports images but reports otherwise just gets a redundant description (harmless, but costs a call).
• In-memory cache. Descriptions are cached by image hash only for the current process; the cache resets on restart.

Acknowledgements

This plugin was inspired by and adapts ideas from:

• JochenYang/opencode-vision — image saving + guiding non-vision models via tool calling.
• WeZZard/opencode-vision — dynamic visual-response skill for OpenCode.
• Nous Research — Hermes Function Calling (Hermes Agent) — capability-aware routing and context-aware content substitution.

Development

pnpm install
pnpm build      # bundle into dist/ with tsdown
pnpm typecheck  # type-check the plugin
pnpm test       # integration tests (vitest; boots a real opencode server)

License

MIT