[BUG] MCP server reports llm_available=false and skips the LLM pass when only the OpenAI fallback credential is set

[wernerkasselman-au]

What I'm seeing

I was reading through the MCP server's "honest accounting" fields (the ones added so a caller can tell a static-only scan from a full semantic one), and I think the availability check is wired to the wrong resolver, so it misreports availability, and it disables the LLM pass in a configuration where the CLI would happily run it.

run_scan() decides whether the semantic pass can run by calling resolve_provider_credentials(), which only ever resolves the active provider's credentials (mcp_server.py:77-83). The model the graph actually constructs, though, comes from create_chat_model(), and that one falls back to a plain OpenAI client (OPENAI_API_KEY / OPENAI_BASE_URL) when the active provider is not configured (providers/__init__.py:120-148). So the gate and the construction path disagree, and the gate is the stricter of the two.

Walking the code (commit 7bc9c0f)

• src/skillspector/mcp_server.py:77-83, the gate, which feeds straight into the state the graph runs on:

  llm_available = resolve_provider_credentials() is not None
  llm_used = use_llm and llm_available
  state = {..., "use_llm": llm_used}

• resolve_provider_credentials() is active-provider-only (providers/__init__.py:95-101).
• The default NVIDIA provider reads only NVIDIA_INFERENCE_KEY (providers/nv_build/provider.py:49-54), so with just OPENAI_API_KEY set it returns None, and llm_available comes out false.
• There is already a resolver that mirrors the real fallback, resolve_chat_model_credentials() (providers/__init__.py:111-117), which falls through to the OpenAI provider exactly the way create_chat_model() does. The gate just is not using it.

Reproduction

With SKILLSPECTOR_PROVIDER unset (the default NVIDIA path), NVIDIA_INFERENCE_KEY unset, and OPENAI_API_KEY set, call scan_skill (or run_scan(target, use_llm=True)) on any skill.

• What I get: "llm_available": false, "llm_used": false, "scan_mode": "static-only", and the semantic pass never runs, even though the equivalent CLI scan on the same environment does run it.
• What I expected: the OpenAI fallback credential is usable, so llm_available should be true and the pass should run.

Why it matters

The accounting fields quietly tell an agent that a clean static-only result was the best available, when a full scan was right there. An agent gating installs on the verdict acts on a weaker scan than it could have had.

What I think the fix is

Gate availability on resolve_chat_model_credentials() rather than resolve_provider_credentials(), so the MCP server's llm_available lines up with what create_chat_model() will actually do. I might be missing a reason the MCP path was deliberately scoped to the active provider only, so if that gating is intentional please say so on the issue; otherwise I am happy to put up the one-line change with a test.

Related

• OpenAI fallback uses active provider default model #38, the sibling OpenAI-fallback defect (get_chat_model() pairing the OpenAI client with a non-OpenAI default model label). Same corner of the provider code, different bug.
• The broader "do not silently degrade to static-only, say so explicitly" thread (Fix Anthropic provider: strip unsupported numeric-bound keywords from tool schemas #3, Anthropic provider silently falls back to static — LLM analyzers 400 on minimum/maximum in tool schema #4, Stage 2: one failed batch aborts the entire meta-analyzer pass and silently falls back to static #9, and the discussion on feat(analyzer): no detection for SSRF / cloud-metadata credential theft #62). This bug is that principle failing at the MCP layer.