exl3

GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode

Validated GLM-5.3 Flash recipe for 2x NVIDIA RTX PRO 6000 Blackwell 96GB: 262K context, EXL3/TR3, adaptive MTP, tools, and vision.

vLLM recipe: GLM-5.3-Flash EXL3 K2 on one DGX Spark GB10. Native MTP k=2. Measured tok/s.

Serve EXL3 (ExLlamaV3 trellis) quantized models on vLLM fork runtimes — any architecture, mixed per-layer bitrates, composable with source-format non-routed weights

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.

GLM-5.3-Flash on two DGX Sparks: opinionated vLLM serving, tinyGLM gates, and reproducible inference research.

Qwen3.8-27B EXL3 (4.00 bpw) + DFlash2 speculative decoding for ExLlamaV3, validated at 262k context on a 24 GB RTX 3090

High-performance runtime extensions for vLLM.

Production-tuned deployment recipe: GLM-5.3-Flash 320B (EXL3 4-bit) on 2x NVIDIA DGX Spark - 850K context, DFlash2 + adaptive-k speculative decoding, full .env tuning and pitfalls

GLM-5.3-Flash EXL3 on three DGX Sparks (TP3/EP3, 1M context, vision, DFlash2) — jspark3 fork, traps, measurements

LLM inference server for ExLlamaV3 / EXL3, with OpenAI- and Anthropic-compatible APIs optimized for Agent workloads.

DeepSeek-V4-Flash-Vision-Exp (EXL3 MixedK, 256 experts, uncensored) on one NVIDIA DGX Spark with vLLM + sparkinfer: 245,760 context, vision + DSpark speculative decoding, CUDA graphs. Recipe, overlay patches, benchmarks, receipts.

LLM / AI inference server for Windows + NVIDIA (EXL3/ExLlamaV3). OpenAI-compatible API, multi-GPU, Blazor admin. Ollama-like, no Docker.

Reproducible 262K GLM-5.3 Flash Uncensored EXL3+DFlash2 deployment for 2x RTX PRO 6000 Blackwell

Containerized private AI lab: TabbyAPI EXL3 + SillyTavern + Open WebUI + Ollama + SearXNG

GLM-5.3-Flash EXL3 4bpw on 2x NVIDIA DGX Spark (GB10): production recipe, boot ladder, quality gate, benchmarks and lessons learned — reproducible from CLAUDE.md/AGENTS.md

A llama-server-compatible HTTP front for ExLlamaV3: /props, /health, /slots and /v1/chat/completions with llama-server's timings object, so llama.cpp tooling drives an EXL3 model unchanged.

DeepSeek-V4.1-Flash EXL3 3.51 bpw at TP3 on 3x RTX PRO 6000 (SM120): keep config, prefill/decode numbers, the failed attempts, and a reproducible image build.

GLM-5.3 Flash on three DGX Sparks: eight concurrent 500k-token sessions with vision, a measured scheduler A/B against upstream, video enablement, reconstructed build, evidence and explicit limits. Fork and continuation of NNNtrance's recipe.

GLM-5.2 753B MoE at EXL3-TR3 3.0bpw on 4x RTX PRO 6000 (SM120): digest-pinned serving stack, validation suite, results. Upstream: vllm#139 / sparkinfer#49