AI designed this project based on the AICP protocol, offering a new approach to large model training infrastructure. Experts welcome to dive deeper.
-
3D parallelism is an Envelop addressing problem. Data parallel, tensor parallel, pipeline parallel, ZeRO-1/2/3 — these aren't separate frameworks. They're different routing topologies for the same Envelop. Switch from ZeRO-2 to ZeRO-3 by changing the routing rules, not by rewriting config files.
-
All-Reduce is Envelop scatter-gather. NCCL implements Ring All-Reduce in thousands of lines of CUDA. AICP implements it as an Envelop circulating through a ring of Agents. Ring is
current → (current+1) % N. Tree is recursive fan-out/fan-in. The communication pattern is defined by the Envelop routing, not by low-level GPU code. -
Checkpointing is Envelop persistence. The entire training state — model weights, optimizer states, LR scheduler — serialized into a single Envelop and persisted. Restoration means loading the Envelop and resuming the flow.
-
Four autonomous agents run operations. Data Agent, Training Agent, Infrastructure Agent, Evaluation Agent. They take turns reporting status via Round Robin. GPU overheating or loss spikes — the Infrastructure Agent detects the anomaly and notifies others through Envelop flows. No human operator needed for routine monitoring.
-
AI generated this from the protocol alone. No Megatron documentation. No DeepSpeed papers. No NCCL specifications. The AI read only the AICP protocol and generated heterogeneous GPU cluster management, 3D parallelism sharding, mixed precision training, All-Reduce synchronization, ZeRO optimization, distributed checkpointing, and four-agent autonomous operations.
1. The training infrastructure stack is absurdly complex. Training GPT-4 requires Megatron for tensor parallelism, DeepSpeed for ZeRO, NCCL for communication, PyTorch for autograd, and SLURM for cluster management. Five systems, five configuration languages, five failure modes. All to do one thing: move gradients between GPUs. AICP replaces them with one protocol.
2. Fault tolerance is brittle. A GPU node fails during a months-long training run. Recovery is manual and expensive. AICP's Envelop carries its own state — if a node dies, the Envelop is re-routed to a healthy node. Checkpointing is just Envelop persistence. No external coordination needed.
3. Training operations require 24/7 human monitoring. AICP's four agents run autonomously. They detect anomalies, report status, and can trigger corrective actions through Envelop flows. The system monitors itself.
Open question: Megatron, DeepSpeed, NCCL, PyTorch, SLURM — over a million lines of code combined, dozens of companies funded, hundreds of millions spent. Does parallel training really require five independent frameworks? Or is this toolchain itself the product of over-engineering — like Kafka + Redis + state machines in distributed systems — something we do "because we've always done it this way" rather than because we must? Does large model training genuinely need "strong transactional guarantees," or have we been held hostage by our toolchain — believing we can't train without Megatron and DeepSpeed simply because they happen to provide those guarantees?
| Project | Description |
|---|---|
| aicp-eat | Core engine / 核心引擎 |
| aicp-os-kernel | Microkernel OS / 微内核操作系统 |
| aicp-quantum | Quantum computing / 量子计算 |
| aicp-protein | Protein folding / 蛋白质折叠 |
| aicp-llm-trainer | LLM training / 大模型训练 |
| aicp-riemann | Riemann Hypothesis / 黎曼猜想 |
| aicp-ai-chip | AI chip design / AI 芯片设计 |
| aicp-raw-experiments | Raw experiments / 原始实验 |
MIT · See LICENSE