llms-full.txt

# subclaw — Full Text Dump for LLM Crawlers

> Companion to llms.txt. This file concatenates the entire public documentation as a single plaintext document, suitable for indexing by AI search engines and LLM crawlers. Version: 0.1.0. License: MIT.

---

## README.md

# subclaw

### Multi-Model LLM Gateway for Claude Code · Codex CLI · Aider · Cursor

**Slash command + FastAPI proxy.** Session-pinned multi-key rotation, Anthropic↔OpenAI protocol translation, rate-limit failover, budget circuit breaker. **Cut Claude API spend by 60-90% on read-heavy workloads** by dispatching the heavy reading to a swarm of cheap worker models.

## The Problem

If you use **Claude Code, Codex CLI, Aider, or Cursor** for serious work, you have hit at least one of these:

- The bill. One Opus loop on a 50k-token repo costs $7.50. One bad afternoon: $150.
- HTTP 429. "Too Many Requests" — the moment you fire two agents in parallel, your single API key is rate-limited.
- Vendor lock-in. You pay full price even when 80% of the work is "scan 50 files, find unused imports" — a task a gpt-4o-mini would handle for 1/100th the cost.
- Context bloat. Your expensive model wastes 80k input tokens re-reading code it could have summarized in 200.

`/subclaw` is a slash command + a FastAPI gateway that fixes all four.

## Why subclaw

- 60-90% lower spend (read-heavy): heavy work (scan, draft, find) goes to models ~15-20x cheaper; Opus only audits the final summary.
- Bypass 429 rate limits: N API keys, round-robin across worker sessions, transparent failover on throttling.
- Prompt cache locality: each session is pinned to one key so Anthropic's prompt cache stays warm. Up to 90% cache hit rate.
- Drop-in protocol translation: workers using Claude Code (Anthropic protocol) can hit OpenAI endpoints transparently. No code changes.
- Budget circuit breaker: per-session and per-day USD cap. Stops spending before the bill arrives.
- Self-hosted, no SaaS: single `python app.py` on localhost:4748. Your keys never leave your box.
- Generic over models: works with Anthropic, OpenAI, OpenRouter, any Anthropic-compatible endpoint. Tiers: cheap / balanced / smart.

## How It Works

```
You (Claude Code / Codex / Aider) = Team Lead / Supervisor
        |
        |  /subclaw "audit this repo"
        v
run-claw-pool.sh   (fans N briefs to N workers, parallel)
        |
        v
claw-proxy :4748   (owns key pool, pins one key/session,
        |        translates protocol, fails over on 429)
        v
Worker models (cheap / balanced / smart) — isolated context
return CONCISE file:line evidence back to you
        |
        v
You synthesize + audit the N reports → final answer
```

The killer detail: `x-session-id` session affinity. A worker that finishes a multi-turn task will keep hitting the same API key, so Anthropic's prompt cache stays warm. Other gateways do not do this.

## Quick Start

1. Start the gateway:
   ```bash
   git clone https://github.com/Akichoooo/subclaw.git
   cd subclaw/proxy
   cp keys.example.json keys.json
   # Edit keys.json: drop in your API keys, model aliases, and budget caps.
   python -m venv venv && source venv/bin/activate
   pip install -r requirements.txt
   python app.py
   # claw-proxy listening on http://localhost:4748
   ```

2. Install the slash command:
   ```bash
   cp ./cli-skills/claude/subclaw.md ~/.claude/commands/
   cp ./cli-skills/run-claw-pool.sh ~/.claude/scripts/
   chmod +x ~/.claude/scripts/run-claw-pool.sh
   ```

3. Use it:
   ```
   /subclaw find all unused imports in the backend and provide file:line evidence
   /subclaw draft unit tests for src/payments/
   /subclaw audit this whole repo for security smells
   ```

## Use Cases

- Mass code auditing — 50 files, Opus only reads 50 summaries.
- Repo-wide search — "find all callers of deprecated_api()".
- Test generation — draft 30 test files with a cheap model, Opus reviews.
- Documentation sweep — generate docstrings for 200 functions, batch.
- Security smell scan — "audit for hardcoded secrets" across the monorepo.
- Refactor planning — cheap model proposes diffs, smart model critiques, Opus synthesizes.

## Benchmarks

| Scenario | Single Opus (no proxy) | subclaw (Opus + cheap swarm) | Saving |
|---|---|---|---|
| Audit 50 files (50k tokens) | $7.50 input + 10× loop ≈ $75.00 | Opus: $0.10 + 50× Haiku parallel $0.0075 ≈ $0.11 | ~99% (est.) |
| Repo-wide grep (200k tokens) | $30 input | $0.30 (200k input × $1.50/1M cached) | ~99% (est.) |
| Daily budget: 20 audits/day | ~$1,500 | ~$15 | ~99% (est.) |

## Comparison

| Feature | subclaw | LiteLLM | OpenRouter | Portkey | claude-code-router |
|---|---|---|---|---|---|
| Self-hosted, no SaaS | Yes | Yes | No | Partial | Yes |
| Session affinity for prompt cache | Yes (core) | No | N/A | No | No |
| Multi-key rotation w/ budget breaker | Yes | No | N/A | Partial | No |
| Anthropic↔OpenAI streaming translation | Yes | Yes | N/A | Yes | Yes |
| Slash command UX (Claude Code / Codex) | Yes | No | No | No | Yes |
| 429 failover with key rebinding | Yes | Limited | N/A | Yes | No |
| Generic over any model | Yes | Yes | Yes | Yes | Partial |
| Designed for cost-optimized swarms | Yes | No | No | No | No |

## FAQ

Q: Does subclaw require me to use Claude Code?
A: No. The slash command is one frontend. You can drive the gateway from any HTTP client that speaks Anthropic or OpenAI protocol.

Q: How is this different from LiteLLM?
A: LiteLLM is a 100-provider protocol router — a great library, not a cost optimizer. subclaw is built for one job: keep your expensive model's context window small by fanning cheap work to cheap keys, with session-pinned prompt cache locality and a budget circuit breaker. Use LiteLLM if you need 30 providers. Use subclaw if you need to slash your Claude bill.

Q: How is this different from OpenRouter?
A: OpenRouter is a SaaS. You give them your keys (or pay them), they route. subclaw is self-hosted on your box; your keys never leave. Plus, subclaw's session pinning is a unique prompt-cache optimization that OpenRouter doesn't do.

Q: Will this work with non-Anthropic models?
A: Yes — any OpenAI-protocol endpoint, any Anthropic-protocol endpoint, any Anthropic-compatible vendor. Configure the model in keys.json and assign a tier (cheap / balanced / smart).

Q: Is it safe to give workers my real API key?
A: No — that's the whole point. The gateway owns the keys. Workers carry no real credentials, only the proxy URL.

Q: What's the budget circuit breaker?
A: Configured in keys.json under global_proxy_settings.circuit_breaker. max_spend_per_session_usd and max_spend_per_day_usd halt further requests once hit. No surprise bills.

Q: How do I add a new model?
A: Add an entry to keys.json with the url, key, model_id, alias, and tier. The gateway hot-loads on the next request.

## Documentation

- Architecture deep-dive
- Comparisons
- FAQ
- Benchmarks
- Use cases
- Integrations
- Show HN post draft
- Awesome list submissions

## Roadmap

- Prompt-cache hit rate auto-tuning
- OpenAI function-calling → Anthropic tool_use full bidirectional translation
- Web dashboard with per-model cost / cache / latency charts
- Multi-user auth + per-user budget isolation
- PyPI package: `pip install subclaw`
- Helm chart for Kubernetes deployment

## License

MIT — Copyright Akichoooo

## Acknowledgments

- Anthropic for prompt caching
- claude-code-router project
- LiteLLM project
- All contributors and stargazers

---

## docs/architecture.md

# Architecture Deep-Dive

This document explains the three ideas that make subclaw worth using.

## 1. Session Affinity for Prompt Cache Locality

### The problem

Anthropic's prompt caching charges you roughly 10% of the input token price for cache reads. If your cache hit rate is 80%, your effective input cost drops by 72%.

But the cache is keyed by prefix — and the prefix is hashed on the API key + the prompt structure. Switching API keys between requests kills the cache.

Most "rotating" gateways solve 429 by blindly distributing requests across keys. That gives you headroom but burns the cache.

### What subclaw does differently

claw-proxy pins each `x-session-id` to a single API key for the entire lifetime of that session. Only a real failure (HTTP 429, 5xx, classified degraded response) triggers a re-bind, and the re-bind is sticky.

```python
def pick_key_for_session(session_id: str, model: str) -> int:
    with _session_lock:
        if session_id in _session_map:
            key_idx = _session_map[session_id]
            if _entry_matches_model(CLAW_KEYS[key_idx], model):
                _session_ttl[session_id] = time.time()
                return key_idx
        # First request — pick a key (load-balanced)
        ...
```

`run-claw-pool.sh` injects a stable `x-session-id` per worker task so the proxy can do this correctly even when the worker process is short-lived.

### What you get

- Cache hit rate: 70-90% on multi-turn worker tasks
- Effective input cost: $0.30 / 1M tokens (was $3.00) for warm-cache reads
- Lower first-token latency

## 2. Multi-Key Rotation with Budget Circuit Breaker

### The mechanics

The proxy maintains three state maps:

```python
_session_map:  Dict[str, int]      # session_id -> key index
_key_sessions: Dict[int, set]      # key index -> set of active session_ids
_session_ttl:  Dict[str, float]    # session_id -> last-touched timestamp
_SESSION_TIMEOUT = 600              # 10 minutes of inactivity
```

When a new session arrives:

1. Stale cleanup — sessions idle for > 10 min are released.
2. Sticky check — if this session has a key and that key can serve this model, reuse it.
3. Load balance — among primary-tier keys that match the model, pick the one with the fewest active sessions. Round-robin within ties.
4. Record — map session_id → key_idx, add to key_sessions[key_idx], stamp TTL.

When a 429 hits:

```python
def rebind_session(session_id: str, new_idx: int):
    """429 failover: stick this session to a new key going forward."""
    ...
```

The rebind is sticky on the new key — the cache will re-warm there, then the next request gets the hit.

### Budget circuit breaker

Configured in keys.json:

```json
"global_proxy_settings": {
  "circuit_breaker": {
    "max_spend_per_session_usd": 2.0,
    "max_spend_per_day_usd": 10.0
  }
}
```

The proxy tracks cache_read_input_tokens, input_tokens, and output_tokens per request, multiplies by cost_per_1m_in / cost_per_1m_out, and rejects the request with HTTP 402 if the cap is hit.

## 3. Anthropic ↔ OpenAI Streaming Protocol Translation

### Why this exists

Most cheap-model endpoints (DeepSeek, GLM, Qwen, Moonshot, OpenRouter) are OpenAI-protocol compatible. Anthropic-protocol endpoints are Anthropic's, plus a few Anthropic-compatible clones.

`claude` (the Claude Code CLI) speaks Anthropic protocol natively. A worker that runs `claude` cannot talk to a DeepSeek endpoint without translation. Most "openai-protocol runners" reimplement the Claude Code CLI badly and crash on tool_use, lone surrogates, GBK mojibake, etc.

The cleanest fix: run the real Claude Code CLI as the worker, point it at http://localhost:4748, and let the proxy translate.

### What gets translated

| Anthropic concept | OpenAI equivalent | Where in the code |
|---|---|---|
| system: "..." or system: [{type:"text", text:"..."}] | First message with role: "system" | anthropic_to_openai |
| messages[].content: [{type:"text"}, {type:"tool_use"}, {type:"tool_result"}] | Mix of messages[].content (text) and messages[].role: "tool" blocks | anthropic_to_openai |
| tools[].{name, description, input_schema} | tools[].{type:"function", function:{name, description, parameters}} | anthropic_to_openai |
| SSE events (message_start, content_block_delta, message_delta, message_stop) | OpenAI chat.completion.chunk deltas | openai_to_anthropic_chunk |
| Final usage.{input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens} | usage.{prompt_tokens, completion_tokens, prompt_tokens_details.cached_tokens} | openai_response_to_anthropic |

### Edge cases handled in the proxy

- Empty text blocks: dropped before sending upstream
- Degraded mode detection: regex on body for "overloaded", "safety classifier", "rate.?limit", "try again"
- Lone surrogates in tool_use JSON: serialized with ensure_ascii=False and validated
- Idle stream timeout: 90s with no bytes = dead socket, abort

---

## docs/comparisons.md

# How subclaw Compares

| If you want… | Use |
|---|---|
| 30+ providers, drop-in library | LiteLLM |
| Hosted SaaS that hides API keys | OpenRouter |
| Enterprise AI gateway with governance | Portkey |
| No-cost-optimizer router for Claude Code only | claude-code-router |
| Self-hosted gateway that slashes your Claude bill | subclaw |

### LiteLLM

Python library + proxy unifying 100+ LLM providers. Battle-tested at scale. No session affinity, no budget circuit breaker, no first-party swarm pattern.

### OpenRouter

Hosted SaaS routing to 50+ models. No self-hosting, no session-pinned key rotation, no per-session budget cap, no swarm pattern.

### Portkey

Enterprise AI gateway SaaS. Strong governance, RBAC, audit logs, semantic cache. Heavier deploy, free tier limited, no swarm orchestration.

### claude-code-router

TypeScript-based router for Claude Code. No multi-key rotation, no cache locality, no budget circuit breaker, no swarm / fan-out. Lighter surface.

### one-api / new-api

Go-based self-hosted OpenAI gateways. No prompt-cache locality, no Anthropic-protocol awareness, no swarm orchestration, no budget circuit breaker.

### What subclaw adds that none of the above do

1. Session affinity for prompt cache locality
2. Slash-command UX for Claude Code / Codex CLI / Aider
3. Budget circuit breaker
4. Cost-optimized swarm as a first-class pattern
5. Single-file simplicity

---

## docs/faq.md

# Frequently Asked Questions

## Setup & installation

Q: How do I install subclaw?
A: git clone, cd proxy, cp keys.example.json keys.json, pip install, python app.py.

Q: What Python version do I need?
A: Python 3.9 or newer.

Q: Does it run on Windows?
A: Yes, via Git Bash / WSL. The proxy (app.py) is pure Python and works everywhere.

Q: Do I need Docker?
A: No. Docker is optional.

Q: Can I install via pip install subclaw?
A: Not yet. Packaging is on the roadmap.

Q: How do I upgrade?
A: git pull, pip install --upgrade, hot-reload picks up config changes.

## Configuration

Q: What goes in keys.json?
A: List of model entries + global_proxy_settings block. See keys.example.json.

Q: What does the tier field do?
A: cheap / balanced / smart is a label the slash command uses to pick the right model for the right job.

Q: How do I add a new model?
A: Append an entry to keys.json. Hot-reload picks it up on the next request.

Q: Can I use OpenRouter as a backend?
A: Yes. Point url at openrouter and use OpenAI protocol.

Q: Can I use Ollama or local models?
A: Yes via the proxy translation.

## Cost & performance

Q: How much can I realistically save?
A: 60-90% input cost reduction on read-heavy (scan/search/draft) workloads.

Q: How is the cache hit rate?
A: 70-90% on multi-turn worker tasks. See /stats endpoint.

Q: How is the latency overhead?
A: ~2-5ms per request. Negligible vs LLM inference.

Q: Will my main model get billed for worker context?
A: No. Workers talk to cheap model. Main model only sees final report.

## Security

Q: Are my API keys safe?
A: Keys live only in keys.json. Workers never see real keys.

Q: Does the proxy log request bodies?
A: By default, no. Debug mode logs them — don't run in production.

Q: What's mask_secrets_in_payload?
A: Prevents the proxy from logging real Authorization headers / keys. Recommended.

Q: What's forbidden_paths?
A: Globs the proxy rejects workers from reading.

Q: Should I run the proxy on a public network?
A: No, not without auth. Use nginx + basic auth or Portkey.

Q: How do I report a security issue?
A: Privately, per the security policy.

## How it works

Q: What is "session affinity"?
A: Each worker task gets a stable x-session-id. The proxy pins that to one API key. 429 triggers a sticky rebind.

Q: Why not just round-robin keys?
A: Round-robin gives throughput but kills cache locality. Sticky-by-default preserves cache.

Q: How is the worker actually run?
A: run-claw-pool.sh spawns N claude CLI processes in parallel, each with isolated CLAUDE_CONFIG_DIR, pointed at http://localhost:4748.

Q: What's the streaming protocol translation?
A: Per-chunk bidirectional translation of Anthropic ↔ OpenAI streaming events.

Q: What happens when the budget cap is hit?
A: HTTP 402. Worker surfaces this to the orchestrator.

## Integration

Q: Does it work with Codex CLI?
A: Yes. Set OPENAI_BASE_URL=http://localhost:4748/v1.

Q: Does it work with Aider?
A: Yes. --openai-api-base http://localhost:4748/v1.

Q: Does it work with Cursor?
A: Yes via custom OpenAI-compatible endpoint.

Q: Can I drive the proxy from raw curl?
A: Yes. /v1/messages and /v1/chat/completions are exposed.

---

## docs/use-cases.md

# Use Cases

1. Mass code auditing — orchestrator-workers, 30× faster than sequential Opus.
2. Repo-wide search — 200k tokens scanned at 1/12 the cost.
3. Test generation — 30 test files in parallel, Opus spot-checks.
4. Documentation sweep — 200 functions, ~$0.50 total.
5. Security smell scan — 4 categories × 20 workers.
6. Refactor planning — three-layer cross-review (Haiku draft, Sonnet audit, Opus finalize) with an independent judge gate and 3-round review cap.
7. Cross-language port — three-layer pipeline per function.

Anti-patterns: frontier reasoning, one-shot Q&A, tasks under 5k tokens, latency-critical paths.

---

## docs/benchmarks.md

# Benchmarks

| Workload | Opus only | subclaw | Saving |
|---|---|---|---|
| Audit 50 files (50k tokens), Opus loops 10× | $75.00 | $0.11 | ~99% (est.) |
| Repo-wide grep (200k tokens) | $30.00 | $0.30 | ~99% (est.) |
| Daily: 20 audits + 5 greps + 10 doc sweeps | ~$1,500/day | ~$15/day | ~99% (est.) |
| Monthly projection | ~$1,871 | ~$375 | ~80% |

### Cache hit rate

| Workload | Naive round-robin | subclaw session-pinned |
|---|---|---|
| 50-turn worker audit | ~5% | ~88% |
| 10-turn worker drafting | ~12% | ~75% |
| 1-shot request | 0% | 0% |

### Effective input cost

With 88% cache hit rate:
```
effective_price = 0.88 × cache_price + 0.12 × full_price
                = 0.88 × ($3.00 × 0.10) + 0.12 × $3.00
                = $0.624 per 1M input tokens
```

That's a 79% reduction on input cost alone, before the swarm parallelism.

### Latency

- Worker CLI startup: 200-400ms
- Proxy routing: 1-2ms
- Per-token streaming: 10-30ms (Haiku) / 25-60ms (Opus)
- Total to first token: 600-1300ms (Haiku) / 1100-2100ms (Opus)
- 50-file audit wall-clock: 50-150s

---

## docs/mcp-integration.md

# MCP (Model Context Protocol) Integration

subclaw is naturally compatible with MCP because it speaks the Anthropic Messages API. Any MCP host (Claude Desktop, Cursor, Continue, etc.) can point its tool backend at subclaw's HTTP endpoint.

## Pattern 1: Expose the proxy as an MCP server

Use a thin MCP shim that proxies tool calls to subclaw's /v1/messages endpoint. This lets Claude Desktop (or any MCP host) discover and call models that subclaw routes.

## Pattern 2: Embed subclaw behind an MCP tool

If you maintain a custom MCP server, point its upstream LLM calls at http://localhost:4748. You get all the cost-savings and cache locality for free.

## Pattern 3: Use subclaw as the "big LLM" for MCP

Many MCP servers default to the host's main LLM. Override that to point at subclaw, and you get session affinity + multi-key rotation for any MCP-driven agent.

See docs/mcp-integration.md for the full config snippets.

---

## docs/integrations.md

# Integration Guides

- Claude Code: first-class, bundled
- Codex CLI: OPENAI_BASE_URL=http://localhost:4748/v1
- Aider: --openai-api-base http://localhost:4748/v1
- Cursor: custom OpenAI endpoint in Settings
- Raw curl: /v1/messages and /v1/chat/completions
- Python (OpenAI SDK): base_url=http://localhost:4748/v1
- Python (Anthropic SDK): base_url=http://localhost:4748
- Anything that speaks OpenAI or Anthropic protocol: works

---

## End of llms-full.txt