You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> **An AI agent skill implementing Google's TurboQuant compression algorithm** — the data-oblivious vector quantization framework that reduces LLM KV cache memory by **6x** and delivers up to **8x speedup** with **zero accuracy loss**. Compatible with Claude Code, Codex CLI, and all agents supporting the [Agent Skills specification](https://github.com/anthropics/skills).
9
10
11
+
> **Looking for the standalone Python library?** See [turboquant-compression](https://github.com/Ryuketsukami/turboquant-compression) — pip-installable with 27 tests and full documentation.
12
+
10
13
---
11
14
12
15
## What is TurboQuant?
@@ -27,10 +30,6 @@ TurboQuant compresses each KV cache vector from 32-bit floats down to **3-4 bits
|**Retraining needed**| None — works with any pretrained model |
29
32
30
-
### Why It Matters
31
-
32
-
TurboQuant is being called [Google's "DeepSeek moment"](https://techcrunch.com/2026/03/25/google-turboquant-ai-memory-compression-silicon-valley-pied-piper/) — a software-only breakthrough that dramatically reduces the hardware cost of AI inference. It works with any transformer model (Gemma, Mistral, Llama, GPT, Claude) without modification.
33
-
34
33
---
35
34
36
35
## What is This Repo?
@@ -56,20 +55,20 @@ When installed, your AI coding agent can:
> **Note:** These benchmarks use `d=256` for fast CI. Production KV caches typically use `d=4096+`, where relative errors are significantly lower due to concentration-of-measure effects.
TurboQuant's key advantage: **zero calibration data needed**. Other methods require representative data samples to build quantization schemes. TurboQuant works on any vector from any model, immediately.
233
241
@@ -256,12 +264,11 @@ See also:
256
264
257
265
---
258
266
259
-
## Related Resources
267
+
## Related
260
268
261
-
-[Google's TurboQuant announcement](https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/) — Official blog post
269
+
-[turboquant-compression](https://github.com/Ryuketsukami/turboquant-compression) — Standalone pip-installable Python library with 27 tests
262
270
-[Anthropic Agent Skills](https://github.com/anthropics/skills) — The skill specification this repo follows
-[VentureBeat coverage](https://venturebeat.com/infrastructure/googles-new-turboquant-algorithm-speeds-up-ai-memory-8x-cutting-costs-by-50/) — Industry impact
271
+
-[arXiv:2504.19874](https://arxiv.org/abs/2504.19874) — Original TurboQuant paper
0 commit comments