feat: add direct upstream passthrough

.gitignore

@@ -2,6 +2,7 @@
 node_modules/
 .env
 .env.*
+llm-proxy-passthrough.json
 !.env.example
 !.env.*.example
 promotion.md

README.md

@@ -71,6 +71,7 @@ See also: [Security](docs/security.md) · [Comparisons](docs/comparisons.md) ·
 - [Quickstart](#quickstart)
 - [Install](#install)
 - [Configuration](#configuration)
+  - [Direct upstream passthrough](#direct-upstream-passthrough)
 - [Tool calling](#tool-calling)
 - [Using with SDKs and tools](#using-with-sdks-and-tools)
   - [n8n](#n8n)
@@ -190,7 +191,8 @@ curl -o .opencode/plugins/llm-proxy.js \
 | `OPENCODE_LLM_PROXY_ALLOW_PRIVATE_NETWORK` | `false` | Set to `true` to allow browser Private Network Access preflights. |
 | `OPENCODE_LLM_PROXY_REQUEST_TIMEOUT_MS` | `120000` | Total request timeout, from 1 to 3,600,000 ms. |
 | `OPENCODE_LLM_PROXY_IDLE_TIMEOUT_SECONDS` | request timeout + 5 s | Seconds a connection may stay silent before the server closes it, from 0 (disabled) to 255. Defaults to the request timeout plus 5 s, or 0 when that exceeds 255 s. |
-| `OPENCODE_LLM_PROXY_MAX_REQUEST_BYTES` | `1048576` | Maximum JSON request body and embedded data-URL size, up to 100 MiB. |
+| `OPENCODE_LLM_PROXY_MAX_REQUEST_BYTES` | `33554432` | Maximum JSON request body and embedded data-URL size, up to 100 MiB. The 32 MiB default accommodates whole-session compaction prompts. |
+| `OPENCODE_LLM_PROXY_PASSTHROUGH_CONFIG` | `~/.config/opencode/llm-proxy-passthrough.json` | Path to a JSON passthrough config (`baseUrl`, optional `prefix`, `unprefixed`, `apiKey`, `authHeader`). Missing or invalid config disables passthrough. Keep this file local. |
 | `OPENCODE_LLM_PROXY_MAX_CONCURRENT_REQUESTS` | `8` | Maximum active POST requests. |
 | `OPENCODE_LLM_PROXY_MAX_QUEUED_REQUESTS` | `32` | Maximum POST requests waiting for capacity; excess requests receive `503`. |
 | `OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE` | `8` | Max concurrent in-flight requests using [tool calling](#tool-calling). |
@@ -202,7 +204,7 @@ curl -o .opencode/plugins/llm-proxy.js \
 | `OPENCODE_LLM_PROXY_METRICS_ENABLED` | `false` | Set to `true` to expose the authenticated Prometheus endpoint at `GET /metrics`. |
 | `OPENCODE_LLM_PROXY_REMOTE_MEDIA_ENABLED` | `false` | Set to `true` to fetch remote media URLs and convert them to embedded data URLs. Leave disabled unless required. |
 | `OPENCODE_LLM_PROXY_REMOTE_MEDIA_ALLOWED_SCHEMES` | `["https"]` | JSON array of allowed remote URL schemes (`https` and, if explicitly enabled, `http`). HTTPS-only is strongly recommended. |
-| `OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_BYTES` | value of `OPENCODE_LLM_PROXY_MAX_REQUEST_BYTES` (`1048576` by default) | Maximum downloaded bytes per remote media item, up to 100 MiB. |
+| `OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_BYTES` | `1048576` | Maximum downloaded bytes per remote media item, up to 100 MiB. |
 | `OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_ITEMS` | `4` | Maximum remote media downloads in one request, from 0 to 10,000. |
 | `OPENCODE_LLM_PROXY_MAX_MEDIA_ITEMS` | `64` | Maximum total embedded and remote media items in one request. |
 | `OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_REDIRECTS` | `3` | Maximum redirects per remote media download, from 0 to 100. |
@@ -217,6 +219,35 @@ The proxy returns a single choice and cannot forward `stop`, `seed`, `frequency_
 
 When an upstream call fails, the error names the reason the provider gave (for example `Upstream request failed: Free usage limit reached`) instead of a bare `Upstream request failed.` Credentials that a provider echoes back are redacted, and the text is truncated to 500 characters, so the detail is useful without exposing a key or token.
 
+### Direct upstream passthrough
+
+For clients that already speak an upstream's native API, the proxy can forward requests **verbatim** instead of running them through an OpenCode session and tool bridge. This preserves the upstream's own streaming, tool-call, and error behavior.
+
+Create a local config file (`~/.config/opencode/llm-proxy-passthrough.json` by default, or point `OPENCODE_LLM_PROXY_PASSTHROUGH_CONFIG` at another path):
+
+```json
+{
+  "baseUrl": "https://upstream.example.com",
+  "prefix": "upstream/",
+  "unprefixed": false,
+  "apiKey": "optional-upstream-key",
+  "authHeader": "authorization"
+}
+```
+
+- `POST /v1/chat/completions`, `POST /v1/responses`, and `POST /v1/messages` are eligible.
+- A model that starts with `prefix` is forwarded to `baseUrl` with the prefix removed (`upstream/model-a` → `model-a`). Prefix matching always wins.
+- Unprefixed discovery is **opt-in**: with `"unprefixed": true`, a model that the upstream lists in `GET /v1/models` is forwarded unchanged, even without the prefix. Because this can silently shadow a local OpenCode model with the same id, it is off by default.
+- The proxy's own bearer auth and CORS checks still apply. The caller's `Authorization`, `x-api-key`, and `anthropic-api-key` headers are removed before forwarding.
+- If `apiKey` is set, it replaces the caller's credentials. `authHeader` defaults to `authorization`, which is sent as `Bearer <apiKey>`; any other header name (for example `x-api-key`) receives the raw key. The same credential is used for the `GET /v1/models` probe.
+- Hop-by-hop headers (`connection`, `keep-alive`, `proxy-authenticate`, `proxy-authorization`, `te`, `trailer`, `transfer-encoding`, `upgrade`), plus any header named in `Connection`, `host`, `content-length`, and the caller's auth headers are stripped before forwarding.
+- The request body is read with the `OPENCODE_LLM_PROXY_MAX_REQUEST_BYTES` cap **before** forwarding (HTTP 413 when exceeded), including for requests that end up on the normal OpenCode-backed path.
+- Passthrough requests bypass the proxy's concurrency queue and request timeout; the upstream's own limits apply.
+- Responses carry `x-opencode-passthrough: true`.
+- The file is read at plugin start. Keep it local and mode `600`; it may contain an internal endpoint or credential. The default path is outside the repository, and `llm-proxy-passthrough.json` is git-ignored.
+
+Requests for models that match neither rule continue through the normal OpenCode-backed path.
+
 ### Session lifecycle (safe teardown)
 
 Each proxied request runs in a throwaway OpenCode session titled `Proxy: <model-id>` (**"Proxy: " is a reserved title prefix** — do not name your own sessions with it). When a request completes normally, the session is deleted immediately. When a request is abandoned mid-turn (client disconnect, timeout, or error), the session is **intentionally left in place**: deleting it immediately races the OpenCode server's final persist of the aborted turn, which surfaces as `FOREIGN KEY constraint failed` errors in the server log (observed at thousands per day under bulk clients that cancel slow streams, e.g. RAG pipelines with aggressive timeouts).

dist/llm-proxy.js

@@ -2,6 +2,9 @@
 import { fileURLToPath } from "node:url";
 import { Buffer as Buffer2 } from "node:buffer";
 import { timingSafeEqual } from "node:crypto";
+import { readFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { join } from "node:path";
 
 // canonical-messages.js
 function isObject(value) {
@@ -1432,7 +1435,10 @@ function getState() {
 }
 var DEFAULTS = Object.freeze({
   requestTimeoutMs: 12e4,
-  maxRequestBytes: 1024 * 1024,
+  // Coding agents routinely send whole-session compaction prompts above 1 MiB.
+  // Keep a bounded default, but do not reject an otherwise valid request at
+  // the old 1 MiB mark. Override with OPENCODE_LLM_PROXY_MAX_REQUEST_BYTES.
+  maxRequestBytes: 32 * 1024 * 1024,
   maxConcurrentRequests: 8,
   maxQueuedRequests: 32,
   bridgeAcquireTimeoutMs: 1e4,
@@ -1516,14 +1522,87 @@ function loadConfig() {
     remoteMedia: {
       enabled: booleanEnv("OPENCODE_LLM_PROXY_REMOTE_MEDIA_ENABLED"),
       allowedSchemes: jsonArrayEnvDefault("OPENCODE_LLM_PROXY_REMOTE_MEDIA_ALLOWED_SCHEMES", ["https"]),
-      maxBytes: integerEnv("OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_BYTES", maxRequestBytes || 1024 * 1024, { min: 1, max: 100 * 1024 * 1024 }),
+      // Keep the media download budget independent: raising the JSON body cap
+      // must not silently raise how much the proxy will fetch from remote URLs.
+      maxBytes: integerEnv("OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_BYTES", 1024 * 1024, { min: 1, max: 100 * 1024 * 1024 }),
       maxItems: integerEnv("OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_ITEMS", 4, { min: 0, max: 1e4 }),
       maxTotalItems: integerEnv("OPENCODE_LLM_PROXY_MAX_MEDIA_ITEMS", 64, { min: 1, max: 1e4 }),
       maxRedirects: integerEnv("OPENCODE_LLM_PROXY_REMOTE_MEDIA_MAX_REDIRECTS", 3, { min: 0, max: 100 }),
       timeoutMs: integerEnv("OPENCODE_LLM_PROXY_REMOTE_MEDIA_TIMEOUT_MS", 1e4, { min: 1, max: 36e5 })
     }
   };
 }
+var HOP_BY_HOP_HEADERS = [
+  "connection",
+  "keep-alive",
+  "proxy-authenticate",
+  "proxy-authorization",
+  "te",
+  "trailer",
+  "transfer-encoding",
+  "upgrade"
+];
+var RESERVED_AUTH_HEADERS = /* @__PURE__ */ new Set([
+  ...HOP_BY_HOP_HEADERS,
+  "host",
+  "content-length",
+  "content-type"
+]);
+function loadPassthroughConfig() {
+  const path = process.env.OPENCODE_LLM_PROXY_PASSTHROUGH_CONFIG ?? join(homedir(), ".config", "opencode", "llm-proxy-passthrough.json");
+  try {
+    const parsed = JSON.parse(readFileSync(path, "utf8"));
+    if (typeof parsed?.baseUrl !== "string" || parsed.baseUrl.trim() === "") return null;
+    const url = new URL(parsed.baseUrl);
+    if (url.protocol !== "http:" && url.protocol !== "https:") return null;
+    const prefix = typeof parsed.prefix === "string" ? parsed.prefix : "";
+    const unprefixed = parsed.unprefixed === true;
+    if (!prefix && !unprefixed) return null;
+    const apiKey = typeof parsed.apiKey === "string" && parsed.apiKey.trim() !== "" ? parsed.apiKey : void 0;
+    const authHeader = typeof parsed.authHeader === "string" && parsed.authHeader.trim() !== "" ? parsed.authHeader.trim().toLowerCase() : "authorization";
+    if (RESERVED_AUTH_HEADERS.has(authHeader)) return null;
+    try {
+      new Headers({ [authHeader]: apiKey ?? "probe" });
+    } catch {
+      return null;
+    }
+    return {
+      prefix,
+      baseUrl: parsed.baseUrl.replace(/\/+$/, ""),
+      unprefixed,
+      apiKey,
+      authHeader
+    };
+  } catch {
+    return null;
+  }
+}
+function applyUpstreamAuth(headers, config) {
+  if (!config.apiKey) return headers;
+  headers.set(
+    config.authHeader,
+    config.authHeader === "authorization" ? `Bearer ${config.apiKey}` : config.apiKey
+  );
+  return headers;
+}
+function stripHopByHop(headers, source) {
+  for (const token of (source.get("connection") ?? "").split(",")) {
+    const name = token.trim().toLowerCase();
+    if (name) headers.delete(name);
+  }
+  for (const name of HOP_BY_HOP_HEADERS) headers.delete(name);
+  return headers;
+}
+function passthroughHeaders(request, config) {
+  const headers = stripHopByHop(new Headers(request.headers), request.headers);
+  headers.delete("host");
+  headers.delete("content-length");
+  headers.delete("authorization");
+  headers.delete("x-api-key");
+  headers.delete("anthropic-api-key");
+  headers.set("content-type", "application/json");
+  return applyUpstreamAuth(headers, config);
+}
 function commonHeaders(request, config) {
   return {
     "cache-control": "no-store",
@@ -1629,7 +1708,7 @@ function isAuthorized(request, config = loadConfig()) {
 function isPlainObject(value) {
   return Boolean(value && typeof value === "object" && !Array.isArray(value));
 }
-async function readJsonBody(request, maxBytes, signal) {
+async function readTextBody(request, maxBytes, signal) {
   const declared = Number(request.headers.get("content-length"));
   if (Number.isFinite(declared) && declared > maxBytes) {
     throw new ProxyError("Request body is too large.", 413, "request_too_large");
@@ -1667,9 +1746,7 @@ async function readJsonBody(request, maxBytes, signal) {
       }
       chunks.push(value);
     }
-    const body = JSON.parse(Buffer2.concat(chunks.map((chunk) => Buffer2.from(chunk))).toString("utf8"));
-    if (!isPlainObject(body)) throw new ProxyError("Request body must be a JSON object.", 400, "invalid_json");
-    return body;
+    return Buffer2.concat(chunks.map((chunk) => Buffer2.from(chunk))).toString("utf8");
   } catch (error) {
     if (error instanceof ProxyError) throw error;
     throw new ProxyError("Request body must be valid JSON.", 400, "invalid_json");
@@ -1678,6 +1755,17 @@ async function readJsonBody(request, maxBytes, signal) {
     reader.releaseLock();
   }
 }
+async function readJsonBody(request, maxBytes, signal) {
+  const text2 = await readTextBody(request, maxBytes, signal);
+  try {
+    const body = JSON.parse(text2);
+    if (!isPlainObject(body)) throw new ProxyError("Request body must be a JSON object.", 400, "invalid_json");
+    return body;
+  } catch (error) {
+    if (error instanceof ProxyError) throw error;
+    throw new ProxyError("Request body must be valid JSON.", 400, "invalid_json");
+  }
+}
 function createRequestSignal(request, timeoutMs) {
   const controller = new AbortController();
   const onAbort = () => controller.abort(new ProxyError("Request was cancelled.", 499, "cancelled"));
@@ -3105,8 +3193,60 @@ function geminiModelFromPath(pathname) {
   const match = pathname.match(/^\/v1beta\/models\/(.+):(?:generateContent|streamGenerateContent)$/);
   return match ? decodeURIComponent(match[1]) : null;
 }
+var PASSTHROUGH_PATHS = /* @__PURE__ */ new Set(["/v1/chat/completions", "/v1/responses", "/v1/messages"]);
+var UPSTREAM_MODELS_TTL_MS = 6e4;
 function createProxyFetchHandler(client) {
   const config = loadConfig();
+  const passthrough = loadPassthroughConfig();
+  let upstreamModelsCache = null;
+  async function isUpstreamModel(model) {
+    if (!passthrough.unprefixed) return false;
+    const now = Date.now();
+    if (!upstreamModelsCache || now - upstreamModelsCache.at > UPSTREAM_MODELS_TTL_MS) {
+      let ids = /* @__PURE__ */ new Set();
+      try {
+        const response = await fetch(`${passthrough.baseUrl}/v1/models`, {
+          headers: applyUpstreamAuth(new Headers(), passthrough),
+          signal: AbortSignal.timeout(3e3)
+        });
+        if (response.ok) {
+          const data = await response.json();
+          ids = new Set((data?.data ?? []).map((entry) => entry?.id).filter((id) => typeof id === "string"));
+        }
+      } catch {
+      }
+      upstreamModelsCache = { at: now, ids };
+    }
+    return upstreamModelsCache.ids.has(model);
+  }
+  async function proxyPassthrough(request, url, body, upstreamModel) {
+    const upstreamBody = JSON.stringify({ ...body, model: upstreamModel });
+    const headers = passthroughHeaders(request, passthrough);
+    let upstream;
+    try {
+      upstream = await fetch(`${passthrough.baseUrl}${url.pathname}${url.search}`, {
+        method: "POST",
+        headers,
+        body: upstreamBody,
+        signal: request.signal
+      });
+    } catch (error) {
+      const message = error instanceof Error ? error.message : String(error);
+      await safeLog(client, "error", "Passthrough request failed", {
+        error: message,
+        path: url.pathname
+      });
+      return badRequest(`Passthrough request failed: ${message}`, 502, request);
+    }
+    const responseHeaders = stripHopByHop(new Headers(upstream.headers), upstream.headers);
+    responseHeaders.delete("content-encoding");
+    responseHeaders.delete("content-length");
+    responseHeaders.set("x-opencode-passthrough", "true");
+    for (const [key, value] of Object.entries(commonHeaders(request, config))) {
+      responseHeaders.set(key, value);
+    }
+    return new Response(upstream.body, { status: upstream.status, headers: responseHeaders });
+  }
   const handleRequest = async (request) => {
     const url = new URL(request.url);
     const origin = request.headers.get("origin");
@@ -3126,6 +3266,44 @@ function createProxyFetchHandler(client) {
     if (!isAuthorized(request, config)) {
       return unauthorized(request);
     }
+    if (passthrough && request.method === "POST" && PASSTHROUGH_PATHS.has(url.pathname)) {
+      let rawBody;
+      try {
+        rawBody = await readTextBody(request, config.maxRequestBytes, request.signal);
+      } catch (error) {
+        if (error instanceof ProxyError) {
+          return badRequest(error.message, error.status, request, error.code);
+        }
+        return badRequest("Failed to read request body.", 400, request);
+      }
+      let parsedBody;
+      try {
+        parsedBody = JSON.parse(rawBody);
+      } catch {
+        parsedBody = void 0;
+      }
+      if (typeof parsedBody?.model === "string") {
+        if (passthrough.prefix && parsedBody.model.startsWith(passthrough.prefix)) {
+          return proxyPassthrough(
+            request,
+            url,
+            parsedBody,
+            parsedBody.model.slice(passthrough.prefix.length)
+          );
+        }
+        if (passthrough.unprefixed && await isUpstreamModel(parsedBody.model)) {
+          return proxyPassthrough(request, url, parsedBody, parsedBody.model);
+        }
+      }
+      const replayHeaders = new Headers(request.headers);
+      replayHeaders.delete("content-length");
+      request = new Request(request.url, {
+        method: request.method,
+        headers: replayHeaders,
+        body: rawBody,
+        signal: request.signal
+      });
+    }
     const started = Date.now();
     const context = createRequestSignal(request, config.requestTimeoutMs);
     let releaseSlot = () => {

docs/security.md

@@ -68,6 +68,21 @@ When enabled, the fetcher defaults to HTTPS only and converts successful downloa
 
 Do not add `http` to `OPENCODE_LLM_PROXY_REMOTE_MEDIA_ALLOWED_SCHEMES` unless transport security is provided by a trusted environment and the risk is understood. These controls reduce SSRF risk but do not make arbitrary untrusted URL fetching preferable to leaving the feature disabled.
 
+## Direct upstream passthrough
+
+Passthrough is a second, separate egress path. When enabled, requests for matching models are forwarded verbatim to the `baseUrl` in `OPENCODE_LLM_PROXY_PASSTHROUGH_CONFIG` (default `~/.config/opencode/llm-proxy-passthrough.json`). The proxy does not validate that endpoint for SSRF or allowlist reasons: it is an operator-supplied upstream, not user-supplied remote media.
+
+Treat the config file as a secret and keep it local, mode `600`, and outside version control:
+
+- It can contain an internal endpoint and an `apiKey`.
+- The default path is outside the repository, and `llm-proxy-passthrough.json` is git-ignored.
+- The proxy applies the configured credential to both the upstream request and the `GET /v1/models` probe. Caller credentials are never forwarded.
+- Hop-by-hop headers, `host`, `content-length`, and caller auth headers are stripped before forwarding.
+- The request body is read with the `OPENCODE_LLM_PROXY_MAX_REQUEST_BYTES` cap before forwarding, so a client cannot make the proxy buffer an unbounded body.
+- Unprefixed model discovery is opt-in (`"unprefixed": true`) because it can shadow local OpenCode models with the same ids.
+
+Enabling passthrough grants every authenticated proxy client the same upstream access that the configured credential has. Use a dedicated, least-privilege key where the upstream supports one, and keep the proxy bound to localhost or behind a trusted network and firewall.
+
 ## Checklist
 
 - [ ] Localhost binding unless network access is genuinely required
@@ -77,4 +92,5 @@ Do not add `http` to `OPENCODE_LLM_PROXY_REMOTE_MEDIA_ALLOWED_SCHEMES` unless tr
 - [ ] Firewall restricts inbound access to known hosts
 - [ ] Tool-using clients are trusted and reviewed
 - [ ] Remote media remains disabled, or its HTTPS-only limits are reviewed
+- [ ] Passthrough config stays local (mode `600`) and uses a least-privilege upstream key
 - [ ] Tokens never written to logs

eslint.config.js

@@ -17,6 +17,7 @@ export default [
         // Web API globals available in both Node and Bun
         Request: "readonly",
         Response: "readonly",
+        Headers: "readonly",
         URL: "readonly",
         TextEncoder: "readonly",
         ReadableStream: "readonly",