moe-engine is a research-grade infrastructure layer for training large Mixture-of-Experts language models at hyperscale. It is designed around one core constraint: at 10K+ GPUs, nodes die continuously. The system must keep training alive end-to-end — routing correctly, checkpointing durably, and resuming without operator intervention.
machine-learning fault-tolerance pytorch triton moe distributed-training mixture-of-experts sparse-training llm-training production-infrastructure
-
Updated
Jun 26, 2026 - Python