Skip to content

feat(gateway): meet what Claude Desktop's third-party mode asks of a gateway - #203

Merged
fylorn merged 1 commit into
mainfrom
claude/core-claude-desktop-gateway-vn5cvx
Sep 25, 2026
Merged

fylorn merged 1 commit into
mainfrom
claude/core-claude-desktop-gateway-vn5cvx

Conversation

@fylorn

@fylorn fylorn commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Requested by F · project thread

What this changes

Before: Claude Desktop's HEAD /api/hello startup probe got a 401, a converted upstream that went quiet left an Anthropic stream silent until the client gave up, and Anthropic clients got the OpenAI-shaped /v1/models list.

After: the gateway answers /api/hello itself, writes event: ping into Anthropic Messages streams after 15 s of upstream silence, and lists models in the Anthropic shape with display_name and, for Claude ids, anthropic_family_tier.

Why

ThinkWatch Lite is adding one-click adoption of Claude Desktop (third-party inference mode). Desktop and the Claude Code engine it embeds expect, per the Desktop gateway guide and the gateway protocol guide:

Requirement Before Now
HEAD /api/hello warm-up probe fell into the passthrough and was refused with 401 (no key) GET/HEAD → 200, no auth, never forwarded
Keep-alive while the model is silent (client aborts after ~5 min without bytes) only when the upstream is Anthropic and sends its own pings gateway writes event: ping every 15 s of upstream silence when the client speaks Anthropic Messages
GET /v1/models OpenAI list (object, created) for every client Anthropic clients get type, display_name, created_at, has_more, first_id, last_id; Claude ids also get anthropic_family_tier
POST /v1/messages?beta=true, anthropic-beta/anthropic-version verbatim, cache_control kept already worked unchanged, now pinned by tests
/v1/messages/count_tokens already forwarded to Anthropic upstreams (and refused, not converted, for others) unchanged, now pinned by a test with ?beta=true

Pings

  • Only for Anthropic Messages clients receiving SSE with a 2xx status. Other formats have no ping event, so they get nothing.
  • Only between frames: if the upstream stalls in the middle of a frame, no ping is written, so a frame is never split.
  • A ping is not upstream output: it bypasses the body record, the usage sniffer, the output-length meter and the tool wall.
  • The interval is tw_gateway::PING_EVERY (15 s), copied into AppState::ping_every so tests can shorten it.

Models

  • The Anthropic objects are a superset: object and created stay, because clients that only send x-api-key are also read as Anthropic.
  • display_name is derived only from ids whose last / segment is claude-… made of plain alphanumeric parts (claude-sonnet-4-5-20250929 → Claude Sonnet 4.5). Any other id is its own display name, which the client treats as "no name given".
  • anthropic_family_tier is set only when the id contains claude or anthropic and exactly one of opus/sonnet/haiku. Nothing marks a non-Claude model as Claude, and no model is added to the list.
  • supports_1m and is_family_default are not set: both are claims about the deployment that the gateway cannot verify.

How it was verified

  • New crates/tw-gateway/tests/claude_desktop.rs with 8 tests: the probe is answered with 200 for HEAD and GET and never reaches the upstream; /v1/messages?beta=true reaches the upstream with the same path, query, anthropic-beta, anthropic-version and cache_control (including ttl); count_tokens?beta=true is forwarded; the Anthropic listing and the single-model lookup return the same object; OpenAI clients still get the OpenAI list; a converted OpenAI-chat upstream that stalls 700 ms produces pings, and the stream minus the pings is a complete Anthropic stream, usage is 5/2 as reported, and the recorded body has no ping; an Anthropic upstream that stalls mid-frame gets a ping only at the frame boundary, and the stream minus the pings is byte-identical to the upstream's; a Chat Completions client gets no pings.
  • 3 unit tests for name and tier derivation in server/listing.rs.
  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace all passed. ./scripts/smoke.sh: 60 passed, 0 failed.
  • No credential is touched: /api/hello returns before identification and reads nothing from the request, and the ping path writes a constant frame.

Notes for review

🤖 Generated with Claude Code

https://claude.ai/code/session_01UNaZJNf7LaEo1ceW5M9tv2


Generated by Claude Code

…gateway

Claude Desktop in third-party inference mode, and the Claude Code engine
it embeds, expect four things from a gateway that the gateway either
did not do or did only partly:

- A `HEAD /api/hello` connection-warming probe at startup. It fell into
  the passthrough and was refused with 401 for carrying no key. It is
  now answered with 200 by the gateway itself, without a key and
  without reaching an upstream.
- Keep-alive pings. The client counts every byte and gives up on a
  stream that is silent for about five minutes. An Anthropic upstream
  sends its own pings while it thinks; a converted upstream sends
  nothing, so long reasoning looked like a dead connection. When the
  client speaks Anthropic Messages and the upstream has produced
  nothing for 15 seconds, the gateway now writes `event: ping`. Pings
  go only between frames (never into one the upstream left half sent),
  bypass the body record, the usage sniffing and the output meter, and
  are never sent to other formats, which have no such event.
- `/v1/models` in the Anthropic shape. Anthropic clients got the OpenAI
  list. They now get `type`, `display_name`, `created_at`, `has_more`,
  `first_id` and `last_id`, and Claude models get
  `anthropic_family_tier`, which Desktop uses to resolve bare aliases
  such as `sonnet`. The OpenAI fields stay on the Anthropic objects,
  because clients that only send `x-api-key` are also read as
  Anthropic. Names are derived only from ids that read as Claude; any
  other id is its own display name and gets no tier.
- `?beta=true`, `anthropic-beta` and `count_tokens` already worked;
  tests now pin them, including `cache_control` reaching the upstream
  untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UNaZJNf7LaEo1ceW5M9tv2
@fylorn fylorn self-assigned this Sep 25, 2026
@fylorn
fylorn merged commit 6d023cf into main Sep 25, 2026
4 checks passed
@fylorn
fylorn deleted the claude/core-claude-desktop-gateway-vn5cvx branch September 25, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants