S₀ Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models
-
Updated
Apr 8, 2026 - Python
S₀ Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models
Hybrid LLM pre-training framework fusing Multi-Head Latent Attention, Gated Delta Net, DeepSeek MoE, and Multi-Token Prediction — 415M active params, 8.31B tokens, single A100 80GB.
Add a description, image, and links to the gated-delta-net topic page so that developers can more easily learn about it.
To associate your repository with the gated-delta-net topic, visit your repo's landing page and select "manage topics."