<!-- Project Info -->
<a href="https://github.com/xberg-io/liter-llm/blob/main/LICENSE">
<img src="https://img.shields.io/badge/License-MIT-007ec6" alt="License" />
</a>
<a href="https://docs.liter-llm.xberg.io">
<img src="https://img.shields.io/badge/Docs-liter--llm-007ec6" alt="Docs" />
</a>
A lighter, faster, safer universal LLM API client — one Rust core, 14 native language bindings, 143 providers.
liter-llm is a universal LLM API client compiled from the ground up in Rust: one core, 14 native language bindings, and 143 providers. No interpreter, no transitive dependency tree, no supply-chain surface area — and a drop-in OpenAI-compatible proxy plus an MCP server in a single 35 MB binary.
- Compiled Rust core — no
pip installsupply chain, no.pthauto-execution hooks, no runtime dependency tree to compromise. - Secrets stay secret — API keys are wrapped in
secrecy::SecretString: zeroed on drop, redacted in logs, never serialized. - Polyglot from day one — Python, TypeScript, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, and WebAssembly, all thin wrappers over the same Rust core, plus a C/FFI surface for everything else.
- Observability built in — production-grade OpenTelemetry with GenAI semantic conventions, not an afterthought callback system.
- Composable middleware — rate limiting, caching, cost tracking, health checks, and fallback as Tower layers you stack like building blocks.
We credit litellm for proving the category; our provider registry was bootstrapped from theirs. See ATTRIBUTIONS.md.
| Feature | Description |
|---|---|
| 143 providers | OpenAI, Anthropic, Google, AWS Bedrock, Groq, Mistral, Together, Fireworks, DeepSeek, Cohere, and 130+ more — compiled at build time |
| 14 native bindings | Rust, Python, Node.js, Go, Java, Kotlin, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WebAssembly — plus a shared C/FFI surface |
| First-class streaming | SSE and AWS EventStream binary protocol with zero-copy buffers |
| Proxy & MCP server | Drop-in OpenAI-compatible proxy (22 endpoints) and MCP tool server in a 35 MB Docker image |
| Tower middleware | Rate limiting, caching (40+ OpenDAL backends), cost tracking, budget enforcement, health checks, and fallback — all composable |
| Observability | OpenTelemetry with GenAI semantic conventions, cost-tracking spans, and HTTP-level tracing |
| Tool calling | Parallel tools, structured outputs, and JSON-schema validation |
| Search & OCR | Web search across 12 providers, document OCR across 4 |
| TOML configuration | liter-llm.toml auto-discovery, custom providers, cache backends, and middleware config |
| Local LLM support | Ollama, LM Studio, vLLM, llama.cpp, LocalAI, and llamafile via OpenAI-compatible APIs |
⭐ Star this repo to show your support — it helps others discover liter-llm.
Java
Available on Maven Central as dev.kreuzberg.literllm:liter-llm. See Java README for the dependency snippet and current version.
Elixir
Add {:liter_llm, "~> 1.6"} to your mix.exs dependencies. See Elixir README for full documentation.
Swift
Add via Swift Package Manager. See Swift README for full documentation.
Kotlin (Android)
Available on Maven Central as dev.kreuzberg:liter-llm-android. See Kotlin README for the dependency snippet and current version.
Zig
See Zig README for installation and usage.
C/C++ (FFI)
Build from source as part of this workspace. See FFI crate for full documentation.
CLI, Proxy & MCP Server
The liter-llm CLI ships both the OpenAI-compatible proxy and the MCP tool server. Install it any of these ways:
brew install xberg-io/tap/liter-llm
cargo install liter-llm-cli # from crates.io
npx @kreuzberg/liter-llm-cli --help # npm (self-installs the binary)
docker run -p 4000:4000 -e LITER_LLM_MASTER_KEY=sk-your-key ghcr.io/xberg-io/liter-llmThen run the proxy or the MCP server:
liter-llm api --config liter-llm-proxy.toml # OpenAI-compatible proxy (22 endpoints)
liter-llm mcp --transport stdio # MCP tool server (stdio)
liter-llm mcp --transport http --port 3001 # MCP tool server (Streamable HTTP)See the MCP server guide and the proxy guide for transports, routing, virtual keys, and budgets. To use the MCP server inside a coding agent, install the liter-llm plugin (below) — it auto-registers the server, no manual config required.
Install the liter-llm plugin from the xberg-io/plugins marketplace. It ships the liter-llm agent skills (chat, streaming, tools, embeddings across 143 providers) and works with every major coding agent — expand your harness below.
Claude Code
/plugin marketplace add xberg-io/plugins
/plugin install liter-llm@kreuzberg
Codex CLI
/plugins add https://github.com/xberg-io/plugins
Then search for liter-llm and select Install Plugin.
Cursor
Settings → Plugins → Add from URL → https://github.com/xberg-io/plugins, then select liter-llm.
Gemini CLI
gemini extensions install https://github.com/xberg-io/plugins
Factory Droid
droid plugin marketplace add https://github.com/xberg-io/plugins
droid plugin install liter-llm@kreuzberg
GitHub Copilot CLI
copilot plugin marketplace add https://github.com/xberg-io/plugins
copilot plugin install liter-llm@kreuzberg
opencode
Not yet published as an opencode package. Install via any harness above (self-hosted marketplace); opencode support is tracked in xberg-io/plugins.
Full guides, the unified chat() API for every binding, multimodal I/O, the proxy/gateway, and the complete provider list live at docs.liter-llm.xberg.io.
Contributions are welcome! See CONTRIBUTING.md for guidelines.
Join our Discord community for questions and discussion.
- Kreuzberg — document intelligence: text, tables, metadata from 91+ formats with optional OCR.
- Xberg Enterprise — managed extraction API with SDKs, dashboards, and observability.
- crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- html-to-markdown — fast, lossless HTML→Markdown engine.
- liter-llm — universal LLM API client with native bindings for 14 languages and 143 providers.
- tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
- alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
MIT — see LICENSE for details.