Mizan

Open-source AI gateway for controlled access, usage metering, and internal credit accounting.

Mizan lets an admin expose AI provider connections behind virtual API keys, model routes, rate limits, concurrency limits, and ledger-backed credits. The first milestone is backend-first: prove the gateway, metering, wallet, and runtime limit engine before building a large dashboard.

Status

Mizan is ready for a backend/API-focused v0.1.0 release candidate. The release surface includes SQLite-first storage, auth and virtual API keys, provider/model routing, OpenAI-compatible non-streaming and streaming chat, OpenAI-compatible non-streaming responses, usage metering, credit ledger updates, Redis runtime limits, request/admin audit log foundations, RTK CLI baseline tooling, provider auth-mode metadata, and Prometheus gateway metrics.

The long-term contract is:

• Keep API-facing provider output in an OpenAI-compatible shape.
• Support both /v1/chat/completions and /v1/responses through the same normalization path.
• Track provider auth mode (api_key, subscription_cli, browser_session) without changing the client contract.
• Add non-API provider runtime adapters later without changing the public OpenAI-compatible surface.

The stable v0.1.0 boundary is backend/API-first. Non-API runtime adapters for subscription CLI or browser-session providers remain follow-up work; their registration metadata is already represented so they can be added without changing the public OpenAI-compatible client contract.

MVP Scope

• OpenAI-compatible gateway for /v1/chat/completions, /v1/responses, and /v1/models
• Admin-managed upstream connections for API providers and local/non-API auth mode families
• User registration, virtual API keys, model access rules, and usage history
• Credit accounting based on input/output token prices per 1M tokens
• Redis-backed rate limits, concurrency limits, and short-lived usage counters
• Durable request, usage, and credit ledger storage

Core Scope

Mizan core manages internal credits, admin grants, user min/max credit policies, usage charges, provider routing, and access limits. Credits are an internal accounting unit used by the gateway to meter and control usage.

Current Docs

• Product Requirements
• Architecture
• Self-Hosted Distributed Proxy
• Engineering Principles
• MVP Roadmap
• Backend Implementation Plan
• v0.1.0 Release Readiness
• Alpha 1 Readiness
• Runtime Limit Testing
• Alpha Runbook
• RTK Base Strategy
• RTK Baseline Attestation
• Research Notes
• Name Options
• Initial Issue Backlog

Recommended MVP Stack

Use Rust for the first backend implementation, with RTK as the starting base for the CLI proxy and token-saving layer. Do not rebuild command rewriting, command-output filtering, or CLI token-saving from scratch. Mizan should wrap, adapt, or vendor the RTK layer, then build gateway, metering, wallet, and admin APIs around it. Use Redis for fast runtime controls and PostgreSQL for source-of-truth records. We run SQLite by default for phase-0/1 and keep PostgreSQL in the migration model for future production deployment.

Recommended Rust stack:

• tokio for async runtime
• axum for HTTP APIs and streaming gateway routes
• sqlx for SQLite and PostgreSQL
• redis or deadpool-redis for runtime counters and leases
• tower middleware for auth, tracing, timeouts, and limits
• RTK-derived mizan-rtk module for CLI proxying and command-output filtering

Target Architecture

flowchart LR
    Client["User app or AI CLI"] --> Gateway["Mizan gateway"]
    Gateway --> Auth["Virtual API key auth"]
    Auth --> Limits["Redis limits"]
    Limits --> Router["Model router"]
    Router --> Provider["Provider or local model"]
    Provider --> Meter["Usage meter"]
    Meter --> Wallet["Credit ledger"]
    Wallet --> DB["SQLite (default) / PostgreSQL"]

Loading

Core Provides

The open-source core should provide:

• Provider connection registry
• Model routing and pricing rules
• Virtual API keys
• Usage metering
• Credit ledger primitives
• Admin-managed min/max credit policy
• Manual/admin credit grants and adjustments
• Admin and user APIs
• Local/self-hosted deployment

Development

The Rust workspace follows the crate boundaries described in Engineering Principles and Backend Implementation Plan.

Install Rust using the pinned toolchain in rust-toolchain.toml, then run:

cargo fmt --all
cargo check --workspace
cargo test --workspace

Environment variables:

• MIZAN_PROVIDER_SECRET_KEY (required before creating provider connections, used to encrypt provider API keys at rest)
• MIZAN_HTTP_ADDR (default 0.0.0.0:18180)
• DATABASE_URL, MIZAN_DB_MAX_CONNECTIONS, MIZAN_RUN_MIGRATIONS for storage
• REDIS_URL, MIZAN_LIMIT_RPM, MIZAN_LIMIT_TPM, MIZAN_LIMIT_CONCURRENCY, MIZAN_LIMIT_WINDOW_SECONDS, and MIZAN_LIMIT_LEASE_SECONDS for runtime limits
• MIZAN_ADMIN_EMAIL, MIZAN_ADMIN_PASSWORD, MIZAN_ADMIN_ROLE for optional bootstrap

Run the API locally:

cargo run -p mizan-api

Run the self-hosted distributed proxy daemon:

cargo run -p mizan-daemon -- run --config ./mizan-daemon.toml

Minimal mizan-daemon.toml:

control_plane_url = "http://127.0.0.1:18180"
daemon_token_path = "/run/secrets/mizan-daemon-token"
local_provider_url = "http://127.0.0.1:11434/v1"
provider_family = "openai-compatible"
advertised_models = ["llama3.1"]
max_concurrency = 2
region = "local"
labels = ["gpu"]
health_addr = "127.0.0.1:19180"
heartbeat_interval_seconds = 30

Run API, SQLite-backed storage, and Redis with Docker Compose:

docker compose up --build

Run the alpha validation flow with Redis available:

MIZAN_REDIS_URL=redis://127.0.0.1:6379 scripts/limit-smoke.sh
REDIS_URL=redis://127.0.0.1:6379/ scripts/alpha-smoke.sh

License

Apache-2.0. See LICENSE.

Security

Do not commit provider credentials, user secrets, or local agent context. See SECURITY.md.

Contributing

Mizan is in bootstrap, so small focused changes are welcome. See CONTRIBUTING.md and the issue templates before opening a PR.